2

I'm looking for datasets in which any quantity is estimated by an individual person. These are datasets in the tradition of Galton (1907), in which individuals estimated the weight of an ox.

Ideally the individual persons would be making these estimates without consultation with other individuals or resources, but this is not essential.

I need to know the true value for the estimate.

Galton, F. (1907). Vox populi (the wisdom of crowds). Nature, 75, 450-451.

| improve this question | |
  • 1
    Do you also need to know the truth of the underlying question? – sheß Aug 22 '15 at 12:14
  • Yes, I do. That's a good point - I'll edit the OP. – user1205901 - Reinstate Monica Aug 22 '15 at 12:53
3

Online contests may provide a source of this data. For example, I don't understand this tweet, but there are hundreds of replies each with a guess around 900-1000.

enter image description here

There are many Twitter contests like this, and the good thing about Twitter is that you can download tweets with code (via the Twitter Public Search API) and then parse and analyse the results. For example, I could download all the responses to this tweet, and the easily parse out integer numbers.

See here for more details about the API.

In terms of formulating a good search string, I'd start with contest+guess and sort by Top (not Live). With the API you can only search back 1 week or so.

| improve this answer | |
  • 1
    I have that "Mining the Social Web" book you linked and also some basic knowledge of how to use the Twitter API. However, according to stackoverflow.com/questions/2693553/… there is no direct way to download all responses to a particular Tweet. How would you download all the replies? – user1205901 - Reinstate Monica Aug 23 '15 at 5:34
  • 1
    With the GET search endpoint I would first search to:@user and then in the results I would select based on the "in_reply_to_status_id" field in the JSON response, by matching it to the particular tweet. – philshem♦ Aug 23 '15 at 9:15
  • 1
    You wrote: "I could download all the responses to this tweet, and the easily parse out integer numbers." Given that the tweet in question is more than 7 days ago, is it nonetheless possible to download the responses using the API? – user1205901 - Reinstate Monica Aug 23 '15 at 13:35
  • 1
    Not exactly with the Search function, so you'd have to find recent contests. You can download old tweets if you get the user timeline (up to 3600 tweets, or so) or if you know the tweet id. For this exact contest, you can scroll all the way to the bottom, hit Control+U (Firefox) to see the HTML source, and then parse the responses out of the HTML source. This can be done even manually (or grep if you have linux, mac). – philshem♦ Aug 24 '15 at 5:36
  • Curiously, the CTRL-U option doesn't work for this competition. When I scroll to the end and press CTRL-U it only includes data from the very first replies. For example, the final guess (1352 entrants) does not appear. – user1205901 - Reinstate Monica Aug 24 '15 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.