Wikipedia: List of online newspaper archives
Many lack "official" interfaces and may restrict the use of materials (see their copyrights), but those that fit the requirements can be scraped with web services like import.io or Python's Scrapy:
(It also includes Google's scans of old ...
The New York Times has a web-based archive of news stories going back to 1851. Headlines are available without being a paid member. I'm sure other individual media companies will have a similar archive, but I'm only familiar with the NYTimes one.
After searching a time range, we notice that the URL of the results page can be used to create a scraping ...
As far as I know, one of the best data sources for NLP has been the ENRON emails. While this is not a "news" source, it would certainly provide some raw data for you to play around with.
Another source is PubMed. While this doesn't provide you with the actual article information, it does provide you with a lot of abstracts for scientific writing. There have ...
The site radio-browser.info might be what you look for. It is a community-curated, public domain dedicated collection of radio streams from all over the world. As of July 2016, it contains just over 4700 stations.
For each stream, it offers the fields:
stream type (codec, bitratea)
station homepage URL
Material like this are rarely truly open because of copyright concerns. That said, there are academic news article corpora which can be obtained.
Reuters produced a corpus in 2000 which was later transferred to NIST. There are actually multiple corpora with hundreds of thousands of articles. More information about how to obtain them is on this page As far ...
You can use Webhose.io API to integrate the data you are looking for. You can filter for news articles, blog posts, online discussions and much more. By default you get 1,000 monthly API requests for free, each request may return up to 100 posts, so you can access up to 100,000 articles a month for free.
The dataset is not available, it could be useful in several projects and this Sunday evening was boring: three good reasons to scrape this site.
Here is the file.
It contains the nominees and the winners from 1970* to 2016 (two sheets). Before merging both, I would prefer that someone take a look at it to check the data quality.
* I wonder why the ...
Here are some resources for searching across newspaper archival records for "crime stories", as well as some non-open (noopen) data solutions, and some references. resources, and thoughts:
Some databases are newspapers - Enoch Pratt Free Library
Baltimore Sun Archive - 1837-Current (1837-1988 Article Image Search, 1989-Current Article Text ...
There is literally a section in the documentation called Bulk Data
we are beginning to provide bulk access to the underlying data sets
To download all bz2 bulk exports you would only need to loop over the URLs in the JSON
Answers to this question are easily found with a search engine. For example,
A list of 789 financial blogs:
Top Personal Finance Blogs
There is no export, but the 7 pages of HTML-tables are pretty readable
<td class="col-counter "></td>
<td class="text col-name"&...
there are a few, certainly not all, dates in this pdf about the history of television. one example: Cable television is introduced in Pennsylvania as a means of bringing television to rural areas. (1948)
not a complete answer, but some information in here for you:
If the dataset you want isn't already available, you can create it with some basic programming. Creating an N-gram tool is pretty straightforward, and the volumes of data are not so large if you limit yourself to one topic or a handful of media sources.
For this type of project, you are looking to cast a wide net and collect as much data as ...
There is a (short) list of Creative-Commons friendly Internet Radio Stations. This meets your license criteria but not the global scope you are probably hoping for. In this list you have a short meta-data description and also URLs. Like all things open and wiki, this list can and should be expanded by the community.
Name: The Radio CC
There is also the All The News dataset from Kaggle.
It has about 140K articles from major USA newspapers and news sites.
Quoting from the Kaggle site:
The publications include the New York Times, Breitbart, CNN, Business Insider, the Atlantic, Fox News, Talking Points Memo, Buzzfeed News, National Review, New York Post, the Guardian, NPR, Reuters, Vox, ...
These are compilations of news headlines by date over long periods of time sourced from 3 very different types of agencies:
ABC Australia 17 years (objective, accurate, reliable, aus, government, active) : https://www.kaggle.com/therohk/million-headlines
Times Of India 19 years (factual, reliable, ind, private, active) :
The Event Registry can provide you access to news articles from last 3 years. The system currently holds over 450 million news articles. You can easily get the access to the data using the Python library available here: https://github.com/EventRegistry/. For each article you can get not only the original metadata but also the list of annotated concepts, ...
It's just a subset of "mainstream media", but you can use the NYTimes API to search for "Correction: ".
They also maintain a page with recent corrections, which point to the amended article.
Their format looks like this, so it should be easy to find with the API.
You'll probably have to create the "database". This NYTimes data would plug into the data ...
It is not clear what you mean by popular news headlines.
Do you mean popular in the sense the article with that headline was shared a lot on social media? If yes, you can use the Event Registry. It provides access to over 100 million news articles and you can sort the articles based on how frequently they were shared on social media. In this way you can ...
Event Registry is a system that provides access to over 100 million news articles published in last 3 years in different languages. You can use the provided API (https://github.com/gregorleban/EventRegistry/) to access the content, search by various criteria for articles, events, concepts, etc.
Wikinews.org is part of the Wikimedia project and uses the standard data dump schema (so that the tools for extracting the Wikipedia data dump can be applied to Wikinews).
General data download page
Search on the index page for enwikinews (en = english)
Today's data dump
Example of the "latest" folder
To get familiar with the content, here ...
Event registry provides access to news articles from more than 100.000 news sources in 15 languages including German. The only problem is that you can only get access to the last 3 years of articles.
You can easily get the access to the data using the Python library available here: https://github.com/gregorleban/EventRegistry/
The US Newspapers Directory has a substantial amount of metadata about publication locations & dates, but nothing on circulation. Some bulk data/linked data access is apparently available.
It's part of the broader Chronicling America site, which offers digitised newspapers from 1836 onwards; 1770 titles, 1.36m issues, 9.55m pages. (Of course, the ...
There is a Pew Research study from July, 2016 on Trust and Accuracy of news media
Few have a lot of confidence in the information they get from professional outlets or friends and family, but large majorities have at least some trust in both; social media gets substantially lower trust scores.