Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
14

While you will find some exceptions, sharing archived tweets is against Twitter Developer Policy. Take all reasonable efforts to do the following, provided that when requested by Twitter, you must promptly take such actions: Delete Content that Twitter reports as deleted or expired; etc If a user deletes their own content, the archive ...


7

Instagram Foursquare Livejournal Flikr Pinterest LinkedIn Blogger Wordpress Hope it'd be enough to begin with.


7

72 hours #gamergate scrape provides a link to download the dataset.


7

scraperwiki lets you search twitter for terms, but also allows twitters search operators. here's a quick search for my handle: https://scraperwiki.com/dataset/crt52ai/view/fpzct4y they also offer a get a list of followers feature, so not exactly what you are looking for, but a twitter tool nonetheless EDIT: some sets i came across: Twitter Data used for ...


6

From the Machine Learning Repository at UCI there are some datasets that may get you started. emails SMS For emails you may also find this StackOverflow discussion helpful, or this 300+MB dataset. For blogs marked as spam, check out this site. Twitter data are not allowed to be publically shared. You can share the analysis or the visualization, but not ...


6

Wikidata contains about 90 000 entries with Twitter username. Also, Wikidata contains about 130 000 entries with MusicBrainz artist ID. Intersecting these sets, one could retrieve about 17 000 twitter accounts of artists, bands etc. SELECT ?agent ?agent_name ?twitter_username ?twitter WHERE { ?agent wdt:P2002 ?twitter_username . ?agent wdt:P434 ?...


5

It's true that historic Twitter data is not open data. I will point out this handy library from Jeremy Singer-Vine, Twick which you can install and configure to harvest from the Twitter API—it's designed for journalists who want to respond quickly to breaking news. FWIW, the availability of that library is why I decided not to vote to close this as off-...


5

You can start with http://everypolitician.org, here's a link to the UK data on github. At Unigraph we've merged everypolitician's with data from Wikidata and Freebase, so you can conveniently query it, for example "get the facebook, twitter and wikidata ids of Adam Holloway, the fb and twitter urls and the party he is member of". Reach to me for an API ...


4

Check out these links below, and you can also search "twitter" in data.world for other related data. There's quite a few users who are already aggregating this data and creating analyses and cool visualizations. If you're interested in collaborating, some are seeking additional contributors to help build out these datasets further. Hope this helps! https://...


4

There are a several ways to get access to historical Twitter data. Here are some options: I've had great luck using Topsy in looking at a wide variety of tweets ranging from disease vectors in Africa to sentiment analysis. Here's the link for 28K #opendata tweets for the last 20 days. You can expand to "all time", search by language, and look at influencers....


4

Have you looked at: http://datasift.com/ it is where I would start. http://gnip.com/ would be another good option.


4

Here's the best open data source for trend data I could think of: Wikipedia page view statistics. Derived datasets: Wikitrends, a daily/weekly/monthly updated list of 10 most popular (as in page views) articles. Bonus: not only absolute, but also relative change top 10 lists (called uptrends/downtrends) are offered. Wikpedia:5000, a weekly updated list of ...


4

Facebook has the Graph API so you can construct your own queries and collect as much data as you want. Otherwise, there are some collected resources described in this thread (warning: from 2010 and maybe involving hacked data). Online Social Neworks from UC Irvine 100 million Facebook pages leaked as torrent If you don't require Facebook, there are much ...


4

My experience has been that you cannot reliably get complete historical tweet data directly from the Twitter API. There are a couple of commercial services who offer exactly that type of service, though neither of them are well set up for one-time usage: Gnip is owned by Twitter, and DataSift provides similar services Other (also commercial) services you ...


4

Getting access to historical Twitter data may end up coming with a price. Here are some options: I've had great luck using Topsy in looking at a wide variety of tweets ranging from disease vectors in Africa to sentiment analysis. Here's the link for 86K #FIFA tweets for the last 30 days. You can expand to "all time", search by language, and look at ...


4

In addition to I.M.'s list, there also exist: Dailymotion Metacafe Google+ Tumblr Youtube (as distinct from G+) Vimeo My employer is building a service to provide a unified API for these services and more, leave a comment if you would like access to our pre-production API. You get access and I get to learn about your use case and add the features that you ...


4

Unfortunately, the search index for searching tweets only goes back 7 days. Link to the documentation here, it can be found under 'until'. You might be able to collect tweets from individual users' timelines, if you have an idea about what users are using those terms in their tweets. You can then use the User Timeline part of the REST API to get up to 3200 ...


4

Wikidata: about 500 UK politicians on Facebook. SELECT ?politician ?politicianLabel ?url ?partyLabel WHERE { ?politician wdt:P2013 ?facebook . ?politician wdt:P27 wd:Q145 . ?politician wdt:P106 wd:Q82955 . OPTIONAL {?politician wdt:P102 ?party} BIND (IRI(CONCAT('https://facebook.com/', ?facebook)) AS ?url) SERVICE wikibase:label { ...


3

If you are patient enough to run a script for a month, you can get region-specific geo-tagged tweets from the API https://stream.twitter.com/1.1/statuses/filter.json?locations=5.955870,45.818020,10.492030,47.808380 where the locations parameter is the bounding box of the region you want. You can see an example python code here*. In terms of sharing ...


3

Depending upon a Twitter user's Twitter configuration you can locate their Twitter profile searching with their email address or telephone number. See more here: https://support.twitter.com/articles/20170001. I'm unaware of a quick way to locate an email address with ONLY a Twitter ID. I suppose someone with a list of valid email addresses could bang away ...


3

The University of California, Berkley has an interactive form for query social survey data they have compiled ('SDA: Survey Documentation and Analysis'). I don't know if they have a REST or other API interface. You can find it at this link: http://sda.berkeley.edu/ Their archives are at this link: http://sda.berkeley.edu/archive.htm The 1972-2010 ...


3

There are also academic surveys. Check ICPSR for open-access survey results.


3

Yes, this information is publicly available from the Twitter REST API. You will need to register an application on https://apps.twitter.com. You can make the calls with Python or R. My R example: library(twitteR) # install.packages("twitteR") # Authenticate api_key <- "a" api_secret <- "b" access_token <- "c" access_token_secret <- "d" # ...


3

It's against Twitter's policy to provide datasets of more than user and tweet IDs. However, as lots of people are tweeting every day, you can construct your own dataset using the twitter api: https://dev.twitter.com/rest/public There are many guides on how to mine tweets, for example: https://marcobonzanini.com/2015/03/02/mining-twitter-data-with-python-...


3

Answers to this question are easily found with a search engine. For example, A list of 789 financial blogs: Top Personal Finance Blogs There is no export, but the 7 pages of HTML-tables are pretty readable <tr class="odd"> <td class="col-counter">601</td> <td class="col-counter "></td> <td class="text col-name"&...


3

Here's another list that's updated more frequently and has slightly different metrics, should also be easy to parse (minus the pagination): http://www.investmentzen.com/top-personal-finance-blogs


2

If you know a bit of Python language it is not too tricky to crawl using the Tweepy library. https://github.com/tweepy/tweepy . It returns sets of tweets in json format. That said it depends on the kind of data you want to use : realtime (stream listener), static, from one or several users, from lists, etc... Every way of crawling has its own limitations due ...


2

I've just been playing with Socrata's site metrics. You get a quite a lot of information, which is nice. Unfortunately, Socrata doesn't have very many users, and its users are probably quite strange compared to the rest of the world.


2

Here is a dataset of relative word frequency for the top 10,000 words in 890 million Tweets, divided by county: https://sites.google.com/site/wordmapperinfo/


2

stackexchange data has practically all of those requirements: http://data.stackexchange.com/stackoverflow/query/new flickr does all of that, although i'm not sure if their use of tags constitutes hashtags, but i'm sure thats easily convertible. you could also use my personal twitter history that i've downloaded. i could careless. maybe you could get a few ...


Only top voted, non community-wiki answers of a minimum length are eligible