5

Can anybody point me to steps how I can get all tweets from #databreach or any #hashtag on Twitter?

The goal is to take all data and export to a file (can be unstructured, structured or semi-structured).

Bonus if usable with the Fabric plugin and Beautiful Soup.

  • Programming question best suited for Android.SX or SE. – philshem Jul 5 '15 at 9:56
  • @philshem Thank you, so I do expect to be related to data science on gathering/collect data step, scala/python/R line of code on getting data, after that i can move with the step in collecting in database or in a file ... the API from twitter was working before and what i did found on google is only related to collecting maximum last 6 days(even that is not working), also are sites what does provide some statistics on using the API | search hashtag but I can't get all tweets related to a hashtag exported to a file ... this type of getting/collect data will help on the creation of DataFrames. – n1tk Jul 5 '15 at 17:10
  • 1
    I'm voting to close this question as off-topic because it belongs on StackOverflow. – Thomas Jul 6 '15 at 15:45
  • I'll vote to leave it open as there is application / software that allow to collect twitter data. – magdmartin Jul 6 '15 at 18:09
  • 1
    Please see the answers to understand why this question is on-topic here. – Nicolas Raoul Jul 14 '15 at 3:23
5

It's true that historic Twitter data is not open data.

I will point out this handy library from Jeremy Singer-Vine, Twick which you can install and configure to harvest from the Twitter API—it's designed for journalists who want to respond quickly to breaking news.

FWIW, the availability of that library is why I decided not to vote to close this as off-topic or as a programming question better suited to Stack Overflow. But I guess that could still happen.

  • so is similar to Streaming Data section from Apache Zeppelin. Thank you. Now I do have the answer on creating data sets. – n1tk Jul 14 '15 at 20:33
4

My experience has been that you cannot reliably get complete historical tweet data directly from the Twitter API. There are a couple of commercial services who offer exactly that type of service, though neither of them are well set up for one-time usage: Gnip is owned by Twitter, and DataSift provides similar services

Other (also commercial) services you might want to look at are Infinigon and Nexalogy. There may be others.

  • Thank you! on historical data I do see the services listed as the only option. – n1tk Jul 7 '15 at 17:28
1

www.followthehashtag.com is a twitter analytics service. You will be able to get Twitter data for free up to 9 days back or track some thousand tweets or, in premium version since 2006, with no total tweets limits (only your budget)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.