Hot answers tagged

4 votes
Accepted

Formats of texts from wikisource?

For automated processing, by far the best alternative is a database dump. They might be a little bit old, but hopefully that won't be a problem. You can select database dumps by language and wiki, ...
user avatar
  • 1,124
3 votes

Where can I find labelled data for handwritten Chinese characters?

This resource has a link to a bunch of online and off line databases for handwritten letters. This offline database seems to be a big one, hosted in Japan by Electrotechnical Laboratory has 1.2 ...
user avatar
3 votes
Accepted

Large English text corpus

Project Gutenberg offers 57.000 free books, available in different formats. Among them, utf-8 encoded plain text with minimal formatting. The NLTK comes with access to a range of corpora. Among them, ...
user avatar
3 votes
Accepted

Looking for datasets of complaints/requests filed at municipalities or cities

US City Open Data Census results for Service Requests should have what you need. And there's always Open 311, though I'm pretty sure its implementers will overlap with the census results. There is ...
user avatar
  • 11.8k
3 votes

Is there any open data-set for sentence segmentation?

The British National Corpus, "a 100 million word collection of samples of written and spoken language from a wide range of sources", is free of charge and segmented, but i never used it.
user avatar
3 votes

Is there any open data-set for sentence segmentation?

Some Corpora in the NLTK have a method sents() that returns each sentence as a list of words, for example the Brown corpus: import nltk nltk.download() # this opens a GUI to download all corpora ...
user avatar
  • 406
2 votes

Actual text and corrected text? Or autocorrect data?

OK, this one is a stretch... but instead of comparing text with errors to the corrected text, you can compare original text against incorrect transcriptions. Reuters Transcribed Subset and data set ...
user avatar
  • 17.2k
2 votes

Formats of texts from wikisource?

If you are just interested in two dozen text I think it is easiest just to request for the raw wikitext, for instance for the 'The Metal Pig' page: https://en.wikisource.org/wiki/The_Metal_Pig?action=...
user avatar
2 votes

List of political blogs

There is a blog that lists political blogs based in the US: http://politicalbloglistings.blogspot.ch/ Categories (count of listed blogs): Conservative (108) Liberal (70) Non/Bi-Partisan or ...
user avatar
  • 17.2k
2 votes

Text of US Presidents' inaugural addresses

Probably not formatted much better than what you've already found (I haven't checked the HTML), but UC-Santa Barbara's American Presidency Project has them as well: http://www.presidency.ucsb.edu/...
user avatar
2 votes
Accepted

Corpus of English text with tagged locations

There are 1,108,558 English Wikipedia articles that have location information. Location information is specified by Template:Coord in Wikipedia and can be easily extracted from the text. I guess ...
user avatar
  • 90
2 votes

Public datasets for Text summarization

I have used both PubMed and Clinical Trials datasets for this. The pubmed is by far the larger set of content with over 1M journals.
user avatar
2 votes
Accepted

Common text data sets in form of panel data

In addition to the answer by @Erwan, I came across nlp-datasets, collected by Nicolas Iderhoff; Awesome Public Datasets, collected by AwesomeData. Both are curated lists of text datasets for NLP ...
user avatar
  • 1,214
2 votes

Common text data sets in form of panel data

It depends what kind of text you're looking for. I can think of the following three sources for text data with years of publication: Project Gutenberg is a repository containing 60,000 books in the ...
user avatar
  • 121
2 votes

Open Source huge dataset for Language Modelling

The Pile An 800GB Dataset of Diverse Text for Language Modeling What is the Pile? The Pile is an 825 GiB diverse, open-source language modeling data set that consists of 22 smaller, high-quality ...
user avatar
  • 1,570
2 votes

Is there a Simplified Chinese thesaurus file, like Moby for English?

The following tools/ websites seems to do what you are looking for: usechinese.com/Apps/Thesaurus (also allows for searching using English input) Chinese-tools. Chinese Synonyms Thesaurus However, ...
user avatar
2 votes

Is there any text data set with both paragraph and document-level labels available?

I think The British National Corpus could do the job. It's a large and free corpus labeled at document, sentence and paragraph level (but also word level, so you'll have a lot of useless xml tags to ...
user avatar
1 vote

Index of YouTube video titles

You can use youtube-dl to make queries, and then dump just the metadata as a json, without downloading the video. Then you can parse the (huge) json to get just the ids and titles and categories, or ...
user avatar
  • 17.2k
1 vote

(Serious) Dataset of paedophilic Youtube comments (or similar)?

Dataset Abusive YouTube Comment Sexually Abusive Comments and specific words collection from popular youtube videos such as music videos and cartoons (Peppa Pig) Trending YouTube Video Statistics and ...
user avatar
  • 1,570
1 vote

computer science corpus for language model training

Could the Semantic Scholar Open Research Corpus (S2ORC) be of help? S2ORC contains 12.7M+ full text papers, of which 7.2% pertain to computer science (according to their paper, at p. 4974).
user avatar
  • 295
1 vote

Corpus suggestion for financial domain

The US Securities and Exchange Commission requires larger publicly traded companies to file quarterly and annual financial statements in XMRL. These can be accessed from the agency's EDGAR site. In ...
user avatar
1 vote

Datasets for Topic Modeling

Julian McAuley of UCSD compiles a list of datasets for recommender systems. These are a diverse set of labelled text data fortified with additional measures available for various datasets (e.g. ...
user avatar
  • 1,214
1 vote

Large English text corpus

I have posted a collection of several text datasets related to news here. They contain 50+ mb of text each in an easily parsable format. https://www.kaggle.com/therohk/datasets
user avatar
  • 141
1 vote

Where can I get conversational/directed sentiment data?

I'm afraid you might be asking too much for what is currently available, in particular the requirement that the sentiment is expressed precisely towards the conversational partner. Maybe you can ...
user avatar
  • 156
1 vote

clinical report dataset

Searching for "clinical science reports" (CSR) on Google returns various websites from pharmaceutical companies, which publish summary reports. Maybe these are enough for your research. As this ...
user avatar
  • 488
1 vote

Is there any text data set with both paragraph and document-level labels available?

You can also use Text8 corpus of Wikipedia text as mentioned in the research paper Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics. I recently worked ...
user avatar
  • 59

Only top scored, non community-wiki answers of a minimum length are eligible