Looking for a database/databases of popular news headlines. These could be data from specific publications or public record, and could be either international or American. Ideally this would cover at least the last 60 years.
The New York Times has a web-based archive of news stories going back to 1851. Headlines are available without being a paid member. I'm sure other individual media companies will have a similar archive, but I'm only familiar with the NYTimes one.
After searching a time range, we notice that the URL of the results page can be used to create a scraping alogirthm.
Unfortunately, it's not a neat and clean database... Here is a sample scraping algorithm I wrote in order to get the Google results count from the HTML page (link). The important part is here (download and parse)
text = requests.get(url).text # get google html m2 = re.search('About ([0-9,]+) results', text) # search for results
You may find that the number of searches is limited (so be gentle with the scraping).
It is not clear what you mean by popular news headlines.
Do you mean popular in the sense the article with that headline was shared a lot on social media? If yes, you can use the Event Registry. It provides access to over 100 million news articles and you can sort the articles based on how frequently they were shared on social media. In this way you can easily find the popular articles. You can use the Python library to freely access the data: https://github.com/gregorleban/EventRegistry/
The system also identifies events mentioned in the articles. In this way it is also easy to detect big events based on the number of different articles that write about them.
These are compilations of news headlines by date over long periods of time sourced from 3 very different types of agencies:
ABC Australia 17 years (objective, accurate, reliable, aus, government, active) : https://www.kaggle.com/therohk/million-headlines
Times Of India 19 years (factual, reliable, ind, private, active) : https://www.kaggle.com/therohk/india-headlines-news-dataset
The Examiner 6 years (crowdsourced, unreliable, clickbait, subjective, usa, private, defunct) : https://www.kaggle.com/therohk/examine-the-examiner