1. Plagiarisms test
The first assignment of this course is to pass the plagiarism test and obtain a certificate at the master and doctoral level. Plagiarism is a serious academic misconduct. You will receive zero grade on plagiarized work and there may be other consequences. We have been told not to do this maybe since primary school, and we are always assuming we know what plagiarism is. However, we may assume we know too much (e.g., famous cases of plagiarism).
You do not need to take this test if you have comparable certification, but the validity of your certification needs to be approved by the instructor.
“All assignments in this course may be processed by TurnItIn, a tool that compares submitted material to an archived database of published work to check for potential plagiarism. Other methods may also be used to determine if a paper is the student’s original work. Regardless of the results of any TurnItIn submission, the faculty member will make the final determination as to whether or not a paper has been plagiarized” (Statement from the Faculty Writing Committee: Guidelines for Preventing Plagiarism).
For this assignment, please submit your certificate as a file via Canvas
2.1 Reading annotation
You are expected to read and annotate the course materials of each week before class (1st week is exempt). The purpose is to have a chance to interact with your classmates, and I will have a better idea of your “pain points” before class. At least have one comment on each article, and respond to at least one comment from another classmate. This regular on-going assignment uses online annotation platform, and is due before the class day.
2.2 In class participation
Your participation is assessed by the instructor and TA based on your performance in class during the lecture, practice, and TA sessions.
2.3 Group project participation
Your participation and responsibilities will be assessed by your group members using this form.
3. Customized learning
You may want to develop your own skill set over the semester. For example, learning how to use Python and R rather than using Stata or Excel. This assignment accommodates such a need. This is a DIY and self-motivated module.
- Learning plan proposal. - Self-evaluation report of completion.
3.1 Learning plan proposal
You are responsible to develop an actionable and feasible learning plan. There is no required structure of the plan, but it should have the following contents:
- An analysis of your background, career goals, and needs of learning.
- Be concrete about your overall learning goals.
- Break down your overall goals into specific and actionable items.
- The plan should cover the major themes of our weekly sessions. Your plan does not need to synchronize with our weekly topics.
- A feasible timeline.
- A concrete plan for assessing your performance.
- Maximum length 2 pages, use the space efficiently and keep your plan concise.
You can share learning plans with others. You can even assemble learning groups. But this is NOT a group assignment, and you will be assessed individually.
You cannot substantially revise your proposal to lower the requirements after it is finalized.
You will assess your own performance using the evaluation plan you proposed. Submit evidence (e.g., a screenshot of completed courses on DataCamp) via Canvas and we may verify.
3.3 Resources for learning
- Free licenses for DataCamp and Brilliant are available via Canvas.
- Stata track: http://geocenter.github.io/StataTraining/
- Other sources: Feel free to help your self!
4. Replication project
Replication is critical to scientific discovery, however, a substantial mount of studies cannot be replicated, leading to the so-called “reproducibility crisis’’ in both natural and social sciences (Baker, 2016; Hardwicke et al., 2020). The replication projects serve two purposes: 1) get familiar with real-word research practices, which are often messy and unclear; 2) practice the empirical methods we learned in class. The reading materials in the first week are important to guiding the replication exercises.
Treat the group replication project as a serious manuscript to be submitted to an academic journal. Set your target high and also be realistic. A good project needs to be a finished project in the first place.
Maximum 4 students per group.
- A replication plan. - A presentation of your replication plan in class. - A replication report. - A presentation of your replication study. - A verification report of another group's replication study.
4.1 Plan your replication
4.1.1 Choosing the right article(s) for replication
You should keep three criteria in mind while searching for the articles for replication (Stojmenovska et al., 2019, p. 305):
- Interest. You are motivated to replicate the study.
- Impact. The article should be published in a disciplinary flagship journal and/or highly cited, making sure you are “learning from the best.”
- Feasibility. Your replication plan should be feasible regarding data access and complexity of analysis.
4.1.2 Replication plan
You can think from five aspects while writing your replication plan: “(1) the premise of the paper, (2) the data and empirical design, (3) the key results, (4) why [you] think the paper would be a good candidate for the replication course, and (5) potential problems that might arise in the replication” (Stojmenovska et al., 2019, p. 306).
A substantial part of the replication plan will go to your final replication report. The contents of the plan may vary, but they should include (but not limited to):
- Significance of replicating the article(s) and your novel contribution.
- Analysis of the original article from: a) theorization, b) research design (e.g., hypothesis and operationalization), c) analysis methods, d) results and conclusions, e) data management and documentation (data management checklist).
- Replication plan: How are you going to replicate and extend the original study? You can develop the plan using the structure for analyzing the original article.
- The following components are required in your replication plan: (a) descriptive statistics, (b) hypothesis testing, (c) regression analysis.
- The following methods are optional in replication: qualitative interviews and coding, additional surveys, computational social science methods.
- Expected results in comparison to original article.
- Knowledge necessary for replicating the article, and how the knowledge is connected to your learning plan (explain by group member). You may not understand all the analysis methods used in the original article, set a plan to learn the new skills.
- Effective teamwork and project management plan. Not all of the members need to be good at running an advanced statistical analysis or designing a rigorous empirical study, but you need to identify your strength and know how to collaborate with and complement each other. This is a more realistic scenario in the future regardless of the career you hope to pursue.
4.1.3 Present your replication plan
Prepare a 10-minute presentation of your replication plan. In the presentation, I expect you will discuss, but are not limited to, the following items:
- Overview of your original study.
- Where you will spend most of your time and why.
- The design of your replication.
- Expected results.
- How it ties to your learning plan.
- Teamwork and division of labor.
Your presentation will be assessed from 1) significance, 2) novel contribution, 3) analysis of the original article, 4) feasibility of replication, 5) teamwork.
4.2 Write and present your replication
Use the published replication studies that are close to your areas of interest as examples, prepare a manuscript of your replication report to be submitted to an academic journal.
Generally, your manuscript should include two parts: a main text published as a regular printed journal article, and an appendix usually published online. The main text summarizes, for example, significance, research design, and findings. The appendix reports all technical details that are necessary for replicating and verifying your work.
Prepare a 15-minute presentation of your replication.
4.3 Verify replications
Your replication report itself needs to be replicable. Each group will be assigned to verify another group’s replication report and provide a verification report. Ideally, your group should be able to replication another group’s replication with the information provided in the replication report. The verification report should point out, for example:
- Whether you can or cannot replicate the replication study, and why.
- Whether the theorization and operationalization are sound.
- Whether the analysis is robust.
- Whether the findings and conclusions are valid.
- How the replication study can be improved.
All groups will revise and finalize their replications using the feedback from the verification reports.
4.4 Replication resources
The replication movement has been progressing very well in psychology, but you should be able to find replication studies in policy and public administration related areas. Here are some resources for your reference:
- Example replication studies.
- Estimating the Reproducibility of Psychological Science
- Mass Replications & Extensions (social psychology)
- “Cited by” articles of the course readings.
5 Late submission
All work is due as indicated on the course schedule. Late submission will be taken a 30% off the total possible points as late penalty. For example, for an assignment with 100 total possible points, if you submit late and graded 80, your final grade for this assignment will be 50 (= 80 − 100 ∗ 30%). Please send me a notice if you have an emergency (neither travel arrangements nor computer problems are emergencies). The last week of class is the final makeup date for all late assignments.
- Baker, M. (2016). 1,500 Scientists Lift the Lid on Reproducibility. Nature News, 533(7604), 452. https://doi.org/10.1038/533452a
- Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An Empirical Assessment of Transparency and Reproducibility-Related Research Practices in the Social Sciences (2014–2017). Royal Society Open Science, 7(2), 190806. https://doi.org/10.1098/rsos.190806
- Stojmenovska, D., Bol, T., & Leopold, T. (2019). Teaching Replication to Graduate Students. Teaching Sociology, 47(4), 303–313. https://doi.org/10.1177/0092055X19867996