4

I'm looking for a dataset of documents (bag of words style is fine) which include time stamps.

I also need it to be semi-coherent, for instance a years worth of news articles which all focus on the economy/finance, broken down by day, or something like that. If it's just a broad collection of documents that aren't really related that doesn't work for me. Something around 30k to 100k documents would be ideal.

I have scoured the internet but haven't really found any datasets of this type besides the NSF abstracts corpus, and unfortunately that doesn't really contain enough in the way of changing temporal dynamics. I need something that's either faster changing (like news), or spans a much larger time interval.

2

The Linguistic Data Consortium (LDC) has an online searchable database of English news articles (also Chinese & Arabic). If you sign up for an account, you should be able to access the LDC Online search. You could retrieve all news articles for a specific topic (e.g., Clinton) and then extract the date.

enter image description here

  • 1
    this looks promising, do you know what the copyright status of these articles are? If I was to create an open source dataset based on some of them is that considered fair use? – Thoth May 21 '16 at 15:01
  • Sorry, I don't know what sort of license issues they use – Brandon Loudermilk May 22 '16 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.