Twitter crawlers for tweets, retweets and social network

Question

I am looking for some twitter crawlers for crawling the tweets/retweets and the social graph of some thousands of users. Suggestions are most welcome.

Answer 1

Have you looked at: http://datasift.com/ it is where I would start. http://gnip.com/ would be another good option.

Answer 2

If you know a bit of Python language it is not too tricky to crawl using the Tweepy library. https://github.com/tweepy/tweepy . It returns sets of tweets in json format. That said it depends on the kind of data you want to use : realtime (stream listener), static, from one or several users, from lists, etc... Every way of crawling has its own limitations due to the Twitter API.

Answer 3

With www.followthehashtag.com twitter analytics service you will be able to get Twitter data for free up to 9 days back or track some thousand tweets or, in premium version since 2006, with no total tweets limits (only your budget)

You can find some million free tweets in “datasets” section. For example, a NASDAQ100 big dataset here, all tweets, about 80 days using companies twitter $Symbol (Ticket with an $)

Hope this helps