I'm trying to get historical twitter data starting from 2010 with specific keywords related to S&P500 companies. I will probably end up using their tickers as keywords once I finalize my list of specific companies.
I'm unable to go back more than a week. Has anyone found a way to get around this? I need to be able to have code available for my research paper/presentation. I've tried TwitterSearch and Tweepy packages.
This is the code using TwitterSearch:
import datetime from TwitterSearch import TwitterSearch, TwitterSearchOrder, TwitterSearchException #the start and end of our twitter data mindate = datetime.date(2010, 01, 01) maxdate = datetime.date(2016, 01, 01) try: tso = TwitterSearchOrder() # create a TwitterSearchOrder object tso.set_keywords(['sp500', 's&p500']) # let's define all words we would like to have a look for tso.set_language('en') # English tweets only tso.set_include_entities(False) # entities... tso.set_until(datetime.date(2016, 01, 01)) #this doesn't work... # it's about time to create a TwitterSearch object with our secret tokens ts = TwitterSearch( consumer_key = 'REMOVED', consumer_secret = 'REMOVED', access_token = 'REMOVED', access_token_secret = 'ALSO REMOVED' ) # this is where the fun actually starts :) for tweet in ts.search_tweets_iterable(tso): print( '\n@%s tweeted: %s' % \ ( tweet['user']['screen_name'], tweet['text']) ) except TwitterSearchException as e: print(e)
I also need to be able to print the date of the tweet, how many likes/retweets it had, and possibly the number of followers the user has. Lastly, I need to be able to save all the data in a csv file.
Any help is greatly appreciated!