[, , ] One of the most exciting claims from Benoit et al. (2015) on crowd-coding of political manifestos is that the results are reproducible. Merz, Regel, and Lewandowski (2016) provides access to the Manifesto Corpus. Try to reproduce Fig 2 from Benoit et al. (2015) using workers from Amazon Mechanical Turk. How similar were your results?
 In the InfluenzaNet project a volunteer panel of people report the incidence, prevalence, and health seeking behavior related to influenza-like-illness (ILI) (Tilston et al. 2010; Noort et al. 2015).
[, , ] The Economist is a weekly news magazine. Create a human computation project to see if the ratio of women to men on the cover has increased over time.
[, , ] Building on the question above, now perform the analysis for all eight regions.
[, ] [Kaggle](https://www.kaggle.com/) is a website that hosts open call projects. Participate in one of those projects.
 Look through a recent issue of a journal in your field. Are there any papers that could have been reformulated as open call projects? Why or why not?
 Purdam (2014) described a distributed data collection about begging in London. Summarize the strengths and weaknesses of this research design.
 Redundancy is an important way to assess the quality of distributed data collection. Windt and Humphreys (2016) develop and test a system to collect reports of conflict events from people in Eastern Congo. Read the paper.
 Karim Lakhani and colleagues (2013) created an open call to solicit new algorithms to solve a problem in computational biology. They received more than 600 submissions containing 89 novel computational approaches. Of the submissions, they received 30 that exceeded the performance of the US National Institutes of Health’s MegaBLAST, and the best submission achieved both greater accuracy and speed (1,000 times faster).
 Many human computation projects rely on participants from Amazon Mechanical Turk. Sign-up to become a worker on Amazon Mechanical Turk. Spend one hour working there. How does this impact your thoughts about the design, quality, and ethics of human compuation projects?