3

I'll be delving into text mining applications for my master thesis and I need data for it. Ideally, I would need a corpora of texts/news articles from some single (or multiple) credible and authoritative source covering financial markets/economy and the like, spanning a time period as long as possible.

It turns out that there is data somewhat what I'm looking for: the Wall Street Journal corpus or Reuters corpus. The problem is that these datasets have many texts for each day, but spanning only few years time.
For my purposes, I would be more interested to have only a few texts/news articles for each day/workday but the whole corpora spanning a decade or more.

Does anyone know of datasets/corporas that would suit my requirements?
Any other possibilities/ideas to construct one such dataset using open resources?

Improve this question
3
  • The problem in my eyes is survivorship bias. Many many companies from the early 2000s don't exist anymore or have been merged with other companies. Do you think it's worth digitising old paper newspapers for the sake of it?
    – hroptatyr
    Jul 16, 2015 at 15:48
  • Other than that, try Bloomberg News. While not open, they have an archive that spans at least 13 years.
    – hroptatyr
    Jul 16, 2015 at 15:50
  • @hroptatyr thats a valid point. I haven't thought about it. though, mostly I guess I am concerned with sources that are already digital, such as bloomberg news. I just have no idea how/what/who I should approach with an inquiry for such resources?
    – Sarunas
    Jul 19, 2015 at 16:15

2 Answers 2

Reset to default
4

Your school's library might have access to databases of newspaper and magazine archives, some going back decades. If you're in a large city, your public library might also have those. The New York Times is one of the most common, but some databases, like ProQuest, have many different archived titles.

To get the articles over decades, you may need to download more than you need and apply some criteria to select only a few per day. The criteria, of course, would depend on your project.

Improve this answer
3

For copyright issues; these datasets are no longer available.

Check out those datasets here:

8.5M titles from Reuters (2007-2016)

https://github.com/philipperemy/Reuters-full-data-set

450k + 100k news from Bloomberg / Reuters (more than a decade)

https://github.com/philipperemy/financial-news-dataset

Improve this answer
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.