Cities with open geotagged Twitter data or Foursquare checkin data?

Question

I'm doing a research project on population dynamics and would love to find a dataset of Foursquare checkins or geotagged tweets within a reasonably large time period (one month or more, ideally not during the winter) in a major city (New York, San Francisco, Boston, etc.). Does anyone know of an open dataset, perhaps originally collected for research purposes, that includes this data?

Answer 1

If you are patient enough to run a script for a month, you can get region-specific geo-tagged tweets from the API

https://stream.twitter.com/1.1/statuses/filter.json?locations=5.955870,45.818020,10.492030,47.808380

where the locations parameter is the bounding box of the region you want. You can see an example python code here*.

In terms of sharing twitter data, the TOS allow you to share the tweet id but not the tweet text, because of users' rights to delete their content. For that reason, many twitter datasets only include tweet ids, and any user of that data set must go collect those tweets on their own.

Despite that, there are some available twitter data sets. See here for more details:

• Twitter open datasets
• FIFA 2014 tweets dataset for academic project

---

*the above code was one part used to make this visualization:

Answer 2

foursquare gave a user a dataset a few years ago, who unexplainedly pulled it offline. i used the wayback machine to save it, and host it on GitHub, although to be honest, i've never gone through it so i'm not sure how much data is there, or what data is there. you can sift through it here:
https://github.com/jalbertbowden/foursquare-user-dataset

Answer 3

Like philshem said, the streaming API will work for Twitter, but it will return a fairly small subset of the stream...the "garden hose", about 10% of the full stream, and an even smaller subset has geo-enabled tweets, reducing the sample to about 3-5%.

For the full (firehose) stream you need to contact GNIP who is an exclusive provider for Twitter. I collected about 1 month for around $500 for my thesis research - it was pretty awesome.

I used both methods, here is my python/tweepy/mysql script to collect the streaming API: geo-tweepy

Answer 4

www.followthehashtag.com has some free datased with twitter geodata, and you can index your own ones

In this url you have some data http://followthehashtag.com/datasets/

Kind regards