3

I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each document, so the main goal of the task is classifying each document to one category such as Sport, Political etc. All what I found in Internet is some small data sets, if anyone can help me to find a big data set that can be used for a such task, I will be very thankful.

Note: all the attributes values should composed of integer number which represent the frequency of word in each document.

edit: I'm looking for CSV file to use it as input for multinomial naive bayes classifier.

  • Are you looking for documents to classify, or documents that have already been classified? Typically, you take whatever you have existing that's been manually classified, and use a portion of that as your training set. (holding some back to use to verify that the classifications are good) – Joe Jan 27 '16 at 1:45
  • Actually, I am looking for a data set that already have been classified, but as I said I need it to be composed of attributes that represent the word occurrences within texts in order to use it to build a Multinomial Naive Classifier. @Joe (sorry but I don't have permission to write comments) – Eyad Jan 27 '16 at 1:53
  • Please specify more clearly what you want to do and what language the texts should be. Note that text is by default copyrighted, if you want free texts for Open Data your choice is restricted. Is wikipedia (using the categories as tags) an option for you? – jknappen Jan 27 '16 at 7:01
  • I think you are looking for Document-Term-Matrices (or TermDocument-Matrices - same thing). You can create these yourself from document collections. Actually, that is the easy part. – knb Jan 27 '16 at 8:59

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.