Appraising public health interventions

Description

The overall process of reviewing evidence on public health interventions involves the following steps:

1. Identify the purpose of the literature review.
2. Find and collate studies to be reviewed (includes articles and evaluation reports).
3. Appraise each article or evaluation report.
4. Formulate a statement on the body of evidence.
5. Publish review findings.
6. Apply review findings to inform decisions about public health policy or practice.

This resource, A Schema for Evaluating Evidence on Public Health Interventions, helps individuals or groups with steps 3 and 4, appraising papers and formulating a summary statement about those articles.

This guide looks at appraisal in two stages:

• appraising individual papers to decide if they provide useful and credible information on an intervention; and
• making conclusions based on the evidence for writing a summary statement about the intervention.

The Schema uses criteria to appraise public health interventions that reflect accepted standards of evaluation research. The guide includes a broad range of questions about:

• the intervention and the setting in which it was implemented;
• the evaluation of the intervention;
• any available reports or papers on the intervention; and
• any evidence on the context for the current review.

Steps for Using Method/Tool

This tool consists of five sections. Each section includes guiding questions and case examples to help the user appraise evidence on public health interventions. Supplementary guides (a checklist of questions) are also available to help in the critical appraisal of included studies. Section 1 is done only once, for the entire review. Sections 2–4 are repeated for each included paper. Section 5 is completed after all included evidence has been appraised, guiding the user in developing a summary of the evidence.

Section 1: Recording the purpose and scope of your review

• State the question you will address in the review.

Section 2: Evaluating each article in the review

• Include publication details.
• Specify the intervention.
• Identify the intervention context.
• Identify the evaluation context—background, purpose and questions asked.
• Identify the methods used to evaluate the intervention.

Section 3: Describing the results from the papers selected

• Delineate the results, groups affected and sustainability of the intervention.

Section 4: Interpreting each paper

• Identify the relevance of the study and its findings to the scope and purpose of your review.

Section 5: Summarizing the body of evidence

• Prepare a collective summary of all papers and reports.
• Group, rate and weigh your papers.
• Formulate a summary statement.

Appendix 1: Hierarchies of study design and designation of levels of evidence
Appendix 2: Supplementary Guides 1 to 5 (Supplementary guides to appraise reviews, randomized controlled trials, observational studies, economic evaluations, qualitative studies)
Appendix 3: A table for summarizing all the papers reviewed