3

I'm looking for a dataset of documents (bag of words style is fine) which include time stamps.

I also need it to be semi-coherent, for instance a years worth of news articles which all focus on the economy/finance, broken down by day, or something like that. If it's just a broad collection of documents that aren't really related that doesn't work for me. Something around 30k to 100k documents would be ideal.

I have scoured the internet but haven't really found any datasets of this type besides the NSF abstracts corpus, and unfortunately that doesn't really contain enough in the way of changing temporal dynamics. I need something that's either faster changing (like news), or spans a much larger time interval.

Improve this question

1 Answer 1

Reset to default
3

The Linguistic Data Consortium (LDC) has an online searchable database of English news articles (also Chinese & Arabic). If you sign up for an account, you should be able to access the LDC Online search. You could retrieve all news articles for a specific topic (e.g., Clinton) and then extract the date.

enter image description here

Improve this answer
2
  • 1
    this looks promising, do you know what the copyright status of these articles are? If I was to create an open source dataset based on some of them is that considered fair use?
    – Set
    Commented May 21, 2016 at 15:01
  • Sorry, I don't know what sort of license issues they use
    – Brandon Loudermilk
    Commented May 22, 2016 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.