I want to create and train a model which classifies a new text content into finance, programming, analytics, design etc. Where can I get enough dataset to train my models?



1 Answer 1


There are plenty of resources out there but you'll have to do a bit of poking around to verify that they contain the specific information you're looking for:

1) The Hathi Trust has available a dataset derived from 4.8 million public domain volumes totaling 1.8 billion pages. The dataset includes over 734 billion words, dozens of languages, and spans multiple centuries. Features are informative, quantified characteristics of a text, and include: * Volume-level metadata * Page-level features * Part-of-speech-tagged token counts * Header and footer identification * Sentence and line count * Algorithmic language detection * Line-level features * Beginning and end line character count * Maximum length of the sequence of capital characters starting a line

sharc [dot] hathitrust [dot] org/features

2) Google's GDELT Project is ambitious in scope. Here's some verbiage from their website, "GDELT monitors the world's broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, emotions, counts, quotes and events driving our global society every second of every day, creating a free open platform for computing on the entire world."

gdeltproject [dot] org/#intro

3) Quandl [dot] com has literally thousands and thousands of indexes available for download

4) You didn't say whether or not you had a budget. Here's a source that is worth looking into but isn't free...

www [dot] wallstreethorizon [dot] com/historical-events-data

Hope this helps...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.