EPID600 (Spring, 2013) Final Paper critique (Exam 3) instructions

Instructions for Final Paper critique (Exam 3)

Please note: clarification questions are welcomed. However, if the answer to your question is in these instructions, a half-point may be charged for the service of having the instructor read the instructions for you! You can avoid the charge by quoting the relevant sentence so it is clear that you have read them.

Students will critique an epidemiologic study selected by the instructors. Critiques will be structured as answers to a set of questions (see below). Each student must work on her/his critique ALONE. Do not communicate, directly or indirectly, with anyone other than an EPID600 instructor about the article, the questions, or the answers. The UNC HONOR CODE APPLIES here (see item B, below, for specifics).

Please read over each of the points below well before the critique is due and re-read them prior to beginning work on it.

ARTICLE to critique: The article or citation to it will be posted in Course Documents / Course Materials (see “Final paper / Article critique - Exam 3”) in Blackboard.

The article for the critique will be announced about a week or so after the first exam is due). You are encouraged to read the article then, re-read it periodically, and make notes for your critique as you learn about the applicable concepts in subsequent classes (e.g., as each module is covered in class, critique that aspect of the study). Since the written critique is meant to draw upon all the concepts that you have studied this semester, we suggest submitting the final version of your critique no more than a week before the due date.

HONOR CODE: In conformity with the UNC HONOR CODE (visit), your critique must be accompanied by the pledge, “On my honor, I have neither given nor received unauthorized aid on this assignment”. You are to work on the critique entirely on your own except as indicated here. You may not assist any other student with her/his critique. Even after you submit your answers you should avoid communicating about your answers or about the article, since some students may be submitting their critiques later. Please inform the instructors of any apparent violation of the Honor Code, since such behavior has a corrosive effect on the community. Instructors are required to report suspected violations to the Student Attorney General and the Dean of Students. Please do not put anyone in the position of having to do that, since it is a very unhappy business. Ask your TA or the faculty for clarifications if you are unclear about anything related to the Honor Code as it applies to this assignment (there is no service charge for Honor Code-related questions even if the information is covered in these instructions).

PERMISSIBLE assistance: Except for material that specifically relates to this assignment (e.g., a student's previously written paper on this article), you may use (with appropriate citation if you quote from or rely on it) any published (print or website) document, including published commentary about the article should you find any. You may also obtain assistance from staff at the UNC Writing Center (www.unc.edu/depts/wcweb/), as long as you inform them of the nature of the assignment and these instructions.

IMPERMISSIBLE assistance: You may not use a student's previously written paper on this article or a grading guide for this assignment. You may not receive assistance from a friend, coworker, spouse, librarian, or anyone other than Dr. Schoenbach or a current teaching assistant even for help reading the article, typing your critique, or proofreading. (Hints may be available for purchase @ 1-2 hpts.) Do not discuss the article or your critique with anyone else.

DUE DATE and EXTENSIONS: Please check the course schedule for the due date. Please note that a critique arriving more than a day or two after the due date, whether or not an extension has been obtained, may result in a temporary course grade of IN (Incomplete) until your final grade is available. The IN grade is intended not as a penalty but to be fair to the graders, since our deadline for submitting grades is entirely inflexible.

LENGTH and FORMAT: Succinctness will be considered in the grading, and each question has a maximum word limit. A half-point will be deducted from the (0-10) score for each answer that exceeds the indicated maximum and another half-point for every additional 50 words beyond the limit. These word limits have been set based on an analysis showing that of past answers conforming to these word limits over half received at least 8 of 10 points and one-fourth received 9 of 10 points. Increasing the limit typically did not materially raise the score distribution. The examination assesses both your judgment and skill in selecting the most important points to include and in presenting them fluently but concisely.

Citations to the assigned article (e.g., for direct quotations or points that the grader might not remember) can be given as page number and column, e.g., “(p795c1)” or “(Table1)” (each of these counts as one "word", because there are no spaces or commas). Other citations should be pasted into the separate text box for each question so that they do not count against the word limits and will be available to the person grading that answer. Number these citations and refer to them by number [1,2] within the text of your answer; include page numbers for direct quotations.

SUBMITTING YOUR CRITIQUE

(We really appreciate your observing these details, since that speeds our handling of the large number of critiques.)

After you compose your critique in a word processing program and print, review, and proofread the final version, please submit your critique using the webform (which will refer to the critique as “Exam 3”).

The form requires your name, PID, and Onyen and will send a copy of your submission to your email address in the UNC directory as of the last roster download – please keep this email(s) until you have received your grade, since if the submission goes awry we will need to grade from your copy so you don’t need to resubmit.

Please be careful to paste each answer into the correct textbox on the form. If you begin each answer with the question number the form can alert you if that answer belongs elsewhere. If you need to correct one or more answers, you may resubmit them up to 24 hours after (a) the submission deadline or (b) the time of your first submission, whichever is later. For each question, later submissions will replace previous ones. You do not need to resubmit answers that have not changed (nor will you see them in the form or in the confirmation email).

Note: Do NOT compose your answers in the webform. Compose them offline and then copy and paste them into the form. Because of the vagaries of electronic submission, be sure to retain your original word processing document.

SCORING

Graders will not know your identity during the grading process. Each of the questions will have equal weight. Each answer will be scored based on the following scale. The overall score for the assignment will be computed as 10 times the average score on the assigned questions.

Not answered: 0% (0 points)
Very poor / Seriously Deficient – reflects very little learning from EPID600 (5-24%: 1-2 out of 10 points)
Poor / Unacceptable – disappointing (25-44%: 3-4 out of 10 points)
Fair / Acceptable – we would not want this work to be regarded as typical of EPID600 graduates (45-64%: 5-6 out of 10 points)
Good / Very Good – we would be pleased for this to be regarded as an example of what students learn in EPID600 (65-84%: 7-8 out of 10 points)
Excellent / Outstanding – we would like to display this as an example of the fine work that EPID600 graduates can do (85-100%: 9-10 out of 10 points)

Answer each question in one (or more) well-constructed paragraph(s), with complete sentences. Although it is suggested that you prepare an outline of the points you want to include, your answers should be written as well-structured paragraphs, i.e., written in complete, grammatically correct sentences, in a logical sequence, with appropriate transitions and good diction (see above, Permissible assistance, for information about the UNC Writing Center; if you wish to use their services it is suggested that you begin well in advance of the due date. In addition, their website has various writing aids). An answer that is, essentially, a list or an outline can earn a maximum score of 9.

Your critique should be carefully proofed (reading it aloud to yourself is a good technique – remember: you should not obtain the assistance of a friend, coworker, or spouse for proofreading). We try to avoid unduly penalizing a critique for minor spelling or grammatical errors if English is not your native language, so you are welcome to indicate (in the citations section) if that is the case. But we do expect you to use a spellchecker.

In their evaluation, the graders will consider accuracy, understanding of the epidemiologic concepts relevant to the article, appropriate use of epidemiologic terminology, and evidence of critical thinking and mastery of epidemiologic concepts. Since knowledgeable epidemiologists can (and do) disagree about the correctness or importance of various aspects of a given study, various answers may be acceptable if well articulated and supported.

For each question, up to 20% of the points (i.e., 2 out of 10) may be awarded for the quality of expression, in terms of logical sequence, clarity, succinctness, diction and usage, sentence structure, grammar and spelling, careful proofing, and adherence to the instructions.

Since different questions will be evaluated by different graders, cross-references are not possible. Thus, if an issue or observation applies to more than one question, you will need to mention it each time. The word limit is designed to allow for necessary repetition across questions.

THE QUESTIONS

Below is a list of 10 questions that provide a framework for critiquing an epidemiologic study. We recommend that you answer all of them, but we will ask you to submit answers for only selected ones. These selected questions will be announced by the end of Exam 2.

By way of background, this examination emulates a task that public health professionals and researchers frequently undertake: analyze and evaluate a research study in order to understand, judge, and interpret what it has found (or not found). We may carry out this task for someone else (e.g., as a research assistant or consultant), for ourselves, or as part of a committee. We often cannot make a full or definitive evaluation ourselves without specialized knowledge. But we do want to understand as much as we can with our present level of knowledge - and more than when we enrolled in EPID600.

A research study typically has a small number of primary research objectives. Common research objectives are to estimate a quantity (e.g., the percentage of a population that uses seatbelts), to describe a phenomenon (e.g., what symptoms characterize illnesses resulting from the pandemic strain of H1N1), or to test a hypothesis (e.g., does passive smoke exposure increase risk of preterm birth). Research studies often require a lot of work and resources (plus journal space and readers' time), so the researcher needs to make a case for the study. The study "rationale" is the reasoning that establishes the importance of the study's objectives and approach. The rationale is a key component of the study, since besides persuading gatekeepers (e.g., reviewers and editors), the rationale provides the scientific and/or public health underpinning for all the decisions made in carrying out the study - and for judging how well it has been carried out. So when you critique an article, you evaluate each aspect in respect to the authors’ research objective(s) and rationale.

A research study builds on what is known and seeks to advance or add to that knowledge. So the investigators should demonstrate that they are aware of what is known, tell us what their study will contribute, and explain how their findings modify or buttress the knowledge base. Thus, studies are judged in relation to where the field is when they were designed and conducted.

PLEASE NOTE: The subquestions (with lower case letters) following each of the numbered questions are provided to help indicate the scope of the numbered questions and as prompts for you and the graders. These subquestions are written for general purpose use, so some may not be relevant or important for this article. Conversely, there may be points relevant for this article that are not included among these specific subquestions.

The questions are:

* 1. Research objective and study rationale: [240 word maximum]

What is the primary research objective of this study?

What is the rationale for this objective [i.e., the reason(s) that it is important to achieve the objective(s) or to answer the research question(s)]?

In terms of public health importance and contribution to knowledge, how strong is the rationale for this study and how well is it presented by the authors (e.g., conceptual framework, supporting evidence, logic)?

How well grounded is the rationale in the published literature (biological, epidemiological)?

* 2. Study design and study population, such as the following: [240 word maximum]

Identify the important design features of the study, such as its basic design, or architecture (e.g., case-control, cohort, etc.) and how the design is implemented (e.g., incident vs. prevalent cases, randomization by group, whether data are collected multiple times, whether follow-up is involved and how it is carried out, etc.])

Compared to other reasonable choices, what are the advantages and disadvantages of this study design and its features for the specific objective(s) or question(s) of this study?

What is the study population for this investigation?

What are the major eligibility criteria (inclusion and/or exclusion criteria)?

How suited is this choice of study population, including eligibility criteria, for the objectives of the study?

* 3. Key variables, measures, and data collection modes, such as the following: [240 word maximum]

What are the key variables and what are their roles (e.g., primary outcome or dependent variable, “exposure” or study factor, major potential confounders and other covariables)?

How are these variables defined and measured?

What are the major modes by which data are being collected? (e.g., self-administered questionnaire, interviewer-administered questionnaire, medical record review, biological specimens, etc.)

How suited are these variables, their definitions, their measurement methods, and the data collection modes for the objectives and rationale of this study? Would other ones have been better for meeting the objectives? If yes, which methods and why?

* 4. Study conduct and quality control, such as the following: [240 word maximum]

How has the study population been recruited (e.g., patients in a clinic, volunteers to advertisements, random digit dialing, area sampling of households, etc.)? If different groups of participants are recruited through different mechanisms (e.g., cases and controls, exposed and unexposed), provide this information for each primary group).

What steps were taken to minimize non-participation and selective factors in recruitment? In retention? How effective were these steps?

How successful, overall, was the data collection? What major steps were taken to improve and document the accuracy of the data collected?

* 5. Data analysis, such as the following: [240 words maximum]

What are the key questions the analysis sought to answer? What was the analytic strategy? What was estimated; what was compared? What considerations did these estimates/comparisons involve?

What are the primary statistical analysis techniques used (e.g., contingency tables, comparison of means, stratified analysis, regression modeling, graphical analysis)? How were they implemented? For example, were the primary variables coded as binary or dichotomous variables, as unordered or ordered categories, as counts, or as continuous measures? What were the primary statistical parameters (measures) estimated, e.g., means, prevalences, incidences, incidence rate ratios, odds ratios, survival curves)?

How well suited were these approaches and choices to the objectives of the study? Do the analytic methods unduly or unnecessarily reduce the number of observations available for analysis? Should other approaches or techniques have been used instead or in addition?

How well do the authors examine possible influences of factors other than the primary ones under study, e.g., factors that may confound or modify the primary variables under study? How well do the authors identify, measure, and control for potentially confounding factors? Do the authors analyze the data separately in respect to a major variable to assess similarities or differences in associations (effect measure modification)? If so, did that lead to important insights? If not, were such analyses needed?

* 6. Findings, such as the following: [240 words maximum]

How many participants are included in the primary analyses; what percentage do they constitute of those who were eligible and selected for inclusion? Have a substantial number of observations been dropped from the analyses?

What are the main findings, including both those related to the primary study question(s) and other important results?

Are there particular results you regard as most important?

How well have the authors reported and presented their findings?

Are there additional results or analyses that you believe should have been reported, data that should have been shown?

Are extraneous results presented?

* 7. Potential concerns in interpreting the findings, such as the following: [240 word maximum]

How completely do the authors account for the disposition of all prospective members of the study population (e.g., persons sampled but not contacted, refusals, exclusions from analysis, etc.)

Does the study population seem to reflect the target population well? What sources of selection bias, if any, are likely to be a problem?

What are the major possible sources of bias and other threats to validity that are important for interpreting the findings?

How well did the authors discuss these threats to validity? Did the authors present them objectively, evaluate their likely importance, and provide evidence in support of that evaluation? Did the authors conduct any specific analyses to evaluate reliability, validity, selection bias, or information bias? What were the results of these analyses?

* 8. Linkage with previous knowledge, including the following: [240 word maximum]

How well did the authors compare their results to the findings from other relevant studies? How well did the authors discuss reasons for differences between previous findings and their own?

How well did the authors evaluate the evidence concerning the study objective or question in regard to possible biological or other mechanisms that could account for their findings and other criteria for causal inference (for this subquestion, please ignore concerns about bias)?

How relevant and responsive to the study rationale was this discussion?

In what ways, if any, have the authors advanced previous knowledge?

* 9. Conclusions, implications, and recommendations, including the following: [240 word maximum]

What are the primary conclusions? Are they stated clearly?

How well are they supported by the findings and discussion?

How directly do the conclusions relate to the primary study objective and rationale?

How well did the authors address implications of their study and/or give insightful recommendations for next steps.

* 10. Overview of strengths and limitations, such as the following: [240 word maximum]

What were the key strengths of this study in regard to its objective and accomplishments?

Has the study taken advantage of new methodology?

Do these strengths or new methodology advance the field? How?

What were the key limitations of this study in regard to its objective(s) and accomplishments?

Are these limitations shared by other studies of this topic?

What would be needed to overcome these limitations?

Please note: Your critique is expected to demonstrate critical thinking in the evaluation of the article. Critical thinking is “the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action.” ("Defining Critical Thinking", a statement by Michael Scriven & Richard Paul for the National Council for Excellence in Critical Thinking Instruction - visit).

Citations: If you use information from the literature in your critique, be sure to provide the complete citation to the source and use quotation marks if appropriate. Plagiarism is a type of scientific misconduct and UNC Honor Code violation that can easily be avoided.

Go to submission form (Note: the submission form will refer to the critique as “Exam 3”.)

Return to EPID600 home page

vschoenb Last updated: 3/23/2012, 7/18/2012, 11/5/2012