I need to download a dataset related to the Umbrella movement in Hong Kong from Twitter and Facebook (tweets, hashtags, followers, contacts, etc.).
-
so you need to write script to pull data from Twiter and Facebok Api basically.– n1tkCommented Jan 14, 2016 at 2:09
-
Thank you very much for your answers. I tried twittercounter.com/pages/buy-stats. I am comfortable with both languages, English and Chinese.– martaCommented Jan 15, 2016 at 2:52
3 Answers
@OCLPHK is the twitter account, try with a wrapper to pull all your data, if don't work than you'll need to buy from GNIP(a twitter company).
Read the twitter documentation for developers, starting here :
https://dev.twitter.com/rest/public/search
and then ask around to find someone who does "web programming" who can automate the twitter-search for you.
At least, skim this documentation.
To do anything useful, you need to register with Twitter's development program, however simple your task might be.
They'll give you (for free) two pairs of looong anonymous username/passwords, "tokens". One pair of tokens is for your registered "app". These are credentials you must provide if you, or your little program, wants to perform an unusual search.
The other pair of tokens is for your account. This way you can configure/use many search-apps, and enable-disable them individually.
You'll see quickly: Asking for followers, favorites etc of users is possible with tokens, (username/pw) only. Moreover it is rate-capped. This means you can ask for the details of 15 followers max, then you must wait for another 15 minutes. So Twitter forces you to think ahead and think thoroughly about what you want to get from their search interface.
You can do mass-downloads but these are more complicated, and less specific (you must do the filtering).
For facebook it's a similar procedure, I guess.
-
Dear all thank you so much for your answers and help. Sb0709 I will try what you said!thanks! All my best.– martaCommented Jan 19, 2016 at 2:51
With www.followthehashtag.com twitter analytics service you will be able to get Twitter data for free up to 9 days back or track some thousand tweets or, in premium version since 2006, with no total tweets limits (only your budget)
You can find some million tweets in “datasets” section (excel).
Best