Below is a list of 10 questions that provide a framework for critiquing an epidemiologic study. We recommend that you answer all of them, but we will ask you to submit answers for selected ones.

Critically reading and evaluating a study is a task that public health professionals and researchers frequently undertake in order to understand, judge, and interpret what a study has found (or not found). We may carry out this task for someone else (e.g., as a research assistant or consultant), for ourselves, or as part of a committee. We often cannot make a full or definitive evaluation ourselves without specialized knowledge. But we do want to understand as much as we can with our present level of knowledge - and more than when we enrolled in EPID600.

A research study typically has a small number of primary research objectives. Common research objectives are to estimate a quantity (e.g., the percentage of a population that uses seatbelts), to describe a phenomenon (e.g., what symptoms characterize illnesses resulting from the pandemic strain of H1N1), or to test a hypothesis (e.g., does passive smoke exposure increase risk of preterm birth).

Research studies often require a lot of work and resources (plus journal space and readers' time), so the researcher needs to make a case for the study. The study "rationale" is the reasoning that establishes the importance of the study's objectives and approach. The rationale is a key component of the study, since besides persuading gatekeepers (e.g., reviewers and editors), the rationale provides the scientific and/or public health underpinning for all the decisions made in carrying out the study - and for judging how well it has been carried out. So when you critique an article, you evaluate each aspect in respect to the authors’ research objective(s) and rationale.

A research study builds on what is known and seeks to advance or add to that knowledge. So the investigators should demonstrate that they are aware of what is known, tell us what their study will contribute, and explain how their findings modify or buttress the knowledge base. Thus, studies are judged in relation to where the field is when they were designed and conducted.

PLEASE NOTE: The bulleted subquestions following each of the numbered questions are provided to help indicate the scope of the numbered questions and as prompts. These subquestions are written for general purpose use, so some may not be relevant or important for this article. Conversely, there may be points relevant for this article that are not included among these specific subquestions.

  1. Research objective and study rationale: [240 word maximum]

• What is the primary research objective of this study?
• What is the rationale for this objective [i.e., the reason(s) that it is important to achieve the objective(s) or to answer the research question(s)]?
• In terms of public health importance and contribution to knowledge, how strong is the rationale for this study and how well is it presented by the authors (e.g., conceptual framework, supporting evidence, logic)?
• How well grounded is the rationale in the published literature (biological, epidemiological)?

  2. Study design and study population, such as the following: [240 word maximum]

• Identify the important design features of the study, such as its basic design, or architecture (e.g., case-control, cohort, etc.) and how the design is implemented (e.g., incident vs. prevalent cases, randomization by group, whether data are collected multiple times, whether follow-up is involved and how it is carried out, etc.])
• Compared to other reasonable choices, what are the advantages and disadvantages of this study design and its features for the specific objective(s) or question(s) of this study?
• What is the study population for this investigation?
• What are the major eligibility criteria (inclusion and/or exclusion criteria)?
• How suited is this choice of study population, including eligibility criteria, for the objectives of the study?

  3. Key variables, measures, and data collection modes, such as the following: [240 word maximum]

• What are the key variables and what are their roles (e.g., primary outcome or dependent variable, “exposure” or study factor, major potential confounders and other covariables)?
• How are these variables defined and measured?
• What are the major modes by which data are being collected? (e.g., self-administered questionnaire, interviewer-administered questionnaire, medical record review, biological specimens, etc.)
• How suited are these variables, their definitions, their measurement methods, and the data collection modes for the objectives and rationale of this study? Would other ones have been better for meeting the objectives? If yes, which methods and why?

  4. Study conduct and quality control, such as the following: [240 word maximum]

• How has the study population been recruited (e.g., patients in a clinic, volunteers to advertisements, random digit dialing, area sampling of households, etc.)? If different groups of participants are recruited through different mechanisms (e.g., cases and controls, exposed and unexposed), provide this information for each primary group).
• What steps were taken to minimize non-participation and selective factors in recruitment? In retention? How effective were these steps?
• How successful, overall, was the data collection? What major steps were taken to improve and document the accuracy of the data collected?

  5. Data analysis, such as the following: [240 words maximum]

• What are the key questions the analysis sought to answer? What was the analytic strategy? What was estimated; what was compared? What considerations did these estimates/comparisons involve?
• What are the primary statistical analysis techniques used (e.g., contingency tables, comparison of means, stratified analysis, regression modeling, graphical analysis)? How were they implemented? For example, were the primary variables coded as binary or dichotomous variables, as unordered or ordered categories, as counts, or as continuous measures? What were the primary statistical parameters (measures) estimated, e.g., means, prevalences, incidences, incidence rate ratios, odds ratios, survival curves)?
• How well suited were these approaches and choices to the objectives of the study? Do the analytic methods unduly or unnecessarily reduce the number of observations available for analysis? Should other approaches or techniques have been used instead or in addition?
• How well do the authors examine possible influences of factors other than the primary ones under study, e.g., factors that may confound or modify the primary variables under study? How well do the authors identify, measure, and control for potentially confounding factors? Do the authors analyze the data separately in respect to a major variable to assess similarities or differences in associations (effect measure modification)? If so, did that lead to important insights? If not, were such analyses needed?

  6. Findings, such as the following: [240 words maximum]

• How many participants are included in the primary analyses; what percentage do they constitute of those who were eligible and selected for inclusion? Have a substantial number of observations been dropped from the analyses?
• What are the main findings, including both those related to the primary study question(s) and other important results?
• Are there particular results you regard as most important?
• How well have the authors reported and presented their findings?
• Are there additional results or analyses that you believe should have been reported, data that should have been shown?
• Are extraneous results presented?

  7. Potential concerns in interpreting the findings, such as the following: [240 word maximum]

• How completely do the authors account for the disposition of all prospective members of the study population (e.g., persons sampled but not contacted, refusals, exclusions from analysis, etc.)
• Does the study population seem to reflect the target population well? What sources of selection bias, if any, are likely to be a problem?
• What are the major possible sources of bias and other threats to validity that are important for interpreting the findings?
• How well did the authors discuss these threats to validity? Did the authors present them objectively, evaluate their likely importance, and provide evidence in support of that evaluation? Did the authors conduct any specific analyses to evaluate reliability, validity, selection bias, or information bias? What were the results of these analyses?

  8. Linkage with previous knowledge, including the following: [240 word maximum]

• How well did the authors compare their results to the findings from other relevant studies? How well did the authors discuss reasons for differences between previous findings and their own?
• How well did the authors evaluate the evidence concerning the study objective or question in regard to possible biological or other mechanisms that could account for their findings and other criteria for causal inference (for this subquestion, please ignore concerns about bias)?
• How relevant and responsive to the study rationale was this discussion?
• In what ways, if any, have the authors advanced previous knowledge?

  9. Conclusions, implications, and recommendations, including the following: [240 word maximum]

• What are the primary conclusions? Are they stated clearly?
• How well are they supported by the findings and discussion?
• How directly do the conclusions relate to the primary study objective and rationale?
• How well did the authors address implications of their study and/or give insightful recommendations for next steps.

10. Overview of strengths and limitations, such as the following: [240 word maximum]

• What were the key strengths of this study in regard to its objective and accomplishments?
• Has the study taken advantage of new methodology?
• Do these strengths or new methodology advance the field? How?
• What were the key limitations of this study in regard to its objective(s) and accomplishments?
• Are these limitations shared by other studies of this topic?
• What would be needed to overcome these limitations?

In your evaluation of the study, try to think critically. Critical thinking is “the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.” ("Defining Critical Thinking", a statement by Michael Scriven & Richard Paul for the National Council for Excellence in Critical Thinking Instruction - visit).

Citations: If you use information from the literature in your critique, be sure to provide the complete citation to the source and use quotation marks if appropriate.