Description
The RE-AIM evaluation framework is one approach to assessing the public health impact of interventions. The overall goal of the framework is to encourage decision makers and others to pay more attention to essential program elements, including external validity, that can improve the implementation of effective and generalizable interventions.
This method provides a way for decision makers to assess how interventions have been implemented in practice, and their subsequent impacts at individual and organizational levels. Specifically, the RE-AIM framework is useful for determining which interventions work in real-world settings and are worth sustained investment.
A number of standard methods are available to determine if an intervention effectively produces the desired outcomes. However, little attention has been paid to examining the potential for translation and the public health impact of interventions. Glasgow and colleagues developed an evaluation framework to expand this assessment of interventions beyond efficacy. Their framework includes multiple criteria dealing with the translatability and public health impact of interventions. This model is an attempt to balance the tension between internal validity (rigour) and external validity (generalizability) of interventions.
The developers expand on Abrams et al's (1996) definition of impact of an intervention as the product of a program's reach (the percentage of the population receiving the intervention) and its efficacy (I = R x E). The framework expands on the "RE" (Reach x Efficacy) concept by adding three dimensions (Adoption, Implementation and Maintenance). The model is one way to assess the trade-off between the intervention's reach and efficacy.
Users can assess the potential public health impact of an intervention using the following five dimensions:
- Reach into the target population (Who will benefit from the intervention?)
- Effectiveness or efficacy (How favourably will the intervention perform in practice?)
- Adoption by target settings, institutions and staff
- Implementation consistency and cost of delivering the intervention (How will the intervention be delivered and received?)
- Maintenance of intervention effects in individuals and settings over time (Does the intervention produce desirable outcomes? How can this be sustained?)
Steps for Using Method/Tool
The RE-AIM framework can be used to plan and evaluate the implementation of public health interventions.
The public health impact of an intervention is a function of five dimensions: Reach, Efficacy, Adoption, Implementation and Maintenance. The current website provides an online method for calculating Reach, Adoption and Impact, and offers additional quizzes and measures.
Reach
Reach is an individual-level measure of participation involving the absolute number, proportion and representativeness of individuals who are willing to participate in a given initiative. Representativeness is defined as the similarity or differences between those who participate and those who are eligible but do not participate. If differences exist, a given intervention may have a differential impact on the population. If differences do not exist, then users can make a stronger case for the generalizabilty of the intervention.
These questions are helpful in assessing the reach of a program:
- What percentage of the target population has come into contact with or has access to your program?
- Will you reach those who are most in need of the intervention?
Effectiveness
Effectiveness reflects the impact of an intervention on important outcomes, such as quality of life and economic outcomes, and considers the potential negative effects of programs. Within the RE-AIM framework, efficacy is measured at the individual level. It reflects the impacts of an intervention when implemented in ideal and real-world settings. Intervention effectiveness is often determined by examining the intervention's effect size and specified outcomes, like quality of life. Outcomes that can be measured include biologic outcomes (e.g., disease risk factors), behavioural outcomes for participants and practitioners and quality-of-life outcomes.
Adoption
Adoption involves the absolute number, proportion and representativeness of settings and intervention agents that will adopt the intervention. It is an organizational level measure. Understanding how different settings vary with respect to available resources, level of expertise and commitment to the intervention is critical when examining the impact of an intervention. If differences exist between participating sites, the program has been differentially adopted. Barriers to adoption can also be determined when examining non-participating settings. Adoption is usually assessed by structured interviews or surveys, or by direct observation.
Implementation
Implementation is the extent to which a program is delivered as intended in the real world. Implementation is an organizational level measure. It includes the consistency of delivery as intended (the fidelity of implementation) and the cost of the intervention.
Maintenance
Maintenance reflects the extent to which an intervention becomes institutionalized or part of routine organizational practices and policies. Maintenance also includes the long-term effects of a program on participants six months or more after the most recent intervention contact. Thus, maintenance involves both individual and organization level measures.
The five dimensions interact to produce a public health impact score or the population-based effect of the intervention. Each of the five components is represented on a 0 to 1 (or 0% to 100%) scale. Multiplying the scores for each dimension yields the public health impact score (R x E x A x I x M = Public Health Impact score). Decision-makers can use this score when considering whether to implement a potential intervention in their setting.
The precise nature of the relationships among the five dimensions are unknown. This model shows these dimensions interacting multiplicatively, rather than in an additive manner. Also, the developers have assumed that the five dimensions are equally weighted. In practice, not all dimensions will be relevant or may be weighted differentially to address a particular public health concern.
Evaluation
RE-AIM is a set of criteria for planning and evaluating interventions, not a theory. It has not been tested for validity or reliability.
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.
We have provided the resources and links as a convenience and for informational purposes only; they do not constitute an endorsement or an approval by McMaster University of any of the products, services or opinions of the external organizations, nor have the external organizations endorsed their resources and links as provided by McMaster University. McMaster University bears no responsibility for the accuracy, legality or content of the external sites.
