[, , , ] One of the most exciting claims from Benoit and colleagues’ (2016) research on crowd-coding of political manifestos is that the results are reproducible. Merz, Regel, and Lewandowski (2016) provides access to the Manifesto Corpus. Try to reproduce figure 2 from Benoit et al. (2016) using workers from Amazon Mechanical Turk. How similar were your results?
 In the InfluenzaNet project a volunteer panel of people report the incidence, prevalence, and health-seeking behavior related to influenza-like-illness (Tilston et al. 2010; Noort et al. 2015).
[, , ] The Economist is a weekly news magazine. Create a human computation project to see if the ratio of women to men on the cover has changed over time.
This question was inspired by a similar project by Justin Tenuto, a data scientist at the crowdsourcing company CrowdFlower: see “Time Magazine Really Likes Dudes” (http://www.crowdflower.com/blog/time-magazine-cover-data).
[, , ] Building on the previous question, now perform the analysis for all eight regions.
[, ] There are several websites that host open call projects, such as Kaggle. Participate in one of those projects, and describe what you learn about that particular project and about open calls in general.
 Look through a recent issue of a journal in your field. Are there any papers that could have been reformulated as open call projects? Why or why not?
 Purdam (2014) describes a distributed data collection about begging in London. Summarize the strengths and weaknesses of this research design.
 Redundancy is an important way to assess the quality of distributed data collection. Windt and Humphreys (2016) developed and tested a system to collect reports of conflict events from people in Eastern Congo. Read the paper.
 Karim Lakhani and colleagues (2013) created an open call to solicit new algorithms to solve a problem in computational biology. They received more than 600 submissions containing 89 novel computational approaches. Of the submissions, 30 exceeded the performance of the US National Institutes of Health’s MegaBLAST, and the best submission achieved both greater accuracy and speed (1,000 times faster).
[, ] Many human computation projects rely on participants from Amazon Mechanical Turk. Sign up to become a worker on Amazon Mechanical Turk. Spend one hour working there. How does this impact your thoughts about the design, quality, and ethics of human computation projects?