Where can I get Twitter datasets available for analysis?
I found two:
While you will find some exceptions, sharing archived tweets is against Twitter Developer Policy.
Take all reasonable efforts to do the following, provided that when requested by Twitter, you must promptly take such actions:
- Delete Content that Twitter reports as deleted or expired;
If a user deletes their own content, the archive should reflect that deletion (which is a massive effort to continuously check).
For that reason, Twitter data sets are often shared as simply two fields: user_id and tweet_id. Then to reconstruct the dataset, one would query the API with those two keys. Then Twitter can ensure that if the tweet was deleted after the initial grab, the content won't show up in the second.
It doesn’t appear that Twitter is changing its terms for developers. Instead, the company has reached an agreement with the Sunlight Foundation and the Open State Foundation, another group involved in versions of Politwoops around the world, to offer the an exception.
If you run the live stream, you can collect your own (you'll get more tweets than you can deal with).
edit: I can add that if you want to 'search' for all tweets, you can use 'lang:en' as your search parameter. See here. Then you can loop over desired langauges.
72 hours #gamergate scrape provides a link to download the dataset.
scraperwiki lets you search twitter for terms, but also allows twitters search operators. here's a quick search for my handle:
they also offer a get a list of followers feature, so not exactly what you are looking for, but a twitter tool nonetheless
EDIT: some sets i came across:
Twitter Data used for sentiment analysis
network Twitter Data
social Twitter data
EDIT 2: Three more sets:
Arizona State University Twitter Data Set
Twitter 2010-10 Dataset
Twitter Data - NIPS 2012
There are a several ways to get access to historical Twitter data. Here are some options:
Check out these links below, and you can also search "twitter" in data.world for other related data. There's quite a few users who are already aggregating this data and creating analyses and cool visualizations. If you're interested in collaborating, some are seeking additional contributors to help build out these datasets further. Hope this helps!
Here is a dataset of relative word frequency for the top 10,000 words in 890 million Tweets, divided by county: https://sites.google.com/site/wordmapperinfo/