Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [corpora]

A database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

1
vote
1answer
16 views

Large English text corpus

I am interested in studying a few specific questions on entropy of different properties of English text. For instance, what is the entropy of capital letters in English? This might tell you something ...
1
vote
0answers
20 views

What are methods to create open text data from copyrighted texts?

Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
0
votes
0answers
19 views

french dataset of short texts with their summaries

I am looking for a french dataset like CNN stories. It will be better if the topic of texts is about university or computer science.
2
votes
0answers
33 views

Audio data set of (telephone) conversations and their transcriptions

I need records of telephone conversations (or simple conversations between two persons) with their transcripts. I need English and French datasets.
0
votes
0answers
37 views

sharing data generated with copyrighted news

This question is related to How can I Share my Data Sets Without Worrying About Copyright Issues?, but asks for the limits of what can still be done. I have a news dataset (scraped from web) that I'd ...
1
vote
1answer
38 views

Books with multiple main characters, one per chapter

I am looking for public domain, or creative commons fiction, with: multiple main characters, written in 3rd person Each chapter focuses on a particular character Modern examples of such books ...
1
vote
0answers
40 views

Licensing of machine learning tool trained on open data

The broad question is: how are the licensing terms for a classifier that is trained with open annotated (or manually annotated) data? I am trying to train a dependency parser for German text with ...
2
votes
2answers
150 views

Swedish Noun Gender List

I'm looking for a lexical resource on Swedish noun genders. In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
1
vote
0answers
61 views

Spanish POS (part of speech) tagged dataset

Where can I get a spanish POS tagged dataset with universal tags free to download?
6
votes
1answer
223 views

Hate speech text data

Where can I find a collection of hate speech articles and texts? I have tried using hate speech and offensive language from github but I am looking for long pieces of text (>~160 characters).
2
votes
1answer
36 views

Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?

I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org manually is quite cumbersome at that scale. Is there a ...
3
votes
2answers
58 views

Corpus of words divided by language

I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language. I don't really care for translations, ...
8
votes
2answers
216 views

Human name country wise corpus

Where I can get the human name corpus country-wise, so that I can use it for training my neural network for detecting the human names out of the string. Does anyone have idea of this?
3
votes
1answer
40 views

Database of Press Releases

Is there a corpus of business/corporate press releases? I'm looking for a collection of releases like these, but from any corporation whatsoever: http://www.businesswire.com/news/home/20161209005816/...
6
votes
1answer
182 views

Actual text and corrected text? Or autocorrect data?

I seek essentially a parallel corpora, which has the dirty original user-generated input on the one side, and a cleaned up version on the other side - with corrected spelling, casing, punctuation and ...
3
votes
1answer
53 views

Is there any picture corpus?

By "picture corpus" I mean the picture collection each illustrating one word in the usually used vocabulary, such as words in Google 10000 English I found on GitHub. For instance: vocabulary ...
3
votes
1answer
104 views

Sentence segmentation dataset

I am looking for datasets that contain texts as well as notations for sentence boundaries. I plan to use it to assess a sentence segmentation system. I am mostly interested in texts in English at the ...
9
votes
4answers
2k views

Corpus of tagged text (English newspapers or any tagged text)

I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks. NOTE: By ...
3
votes
0answers
140 views

Big data set for Document Classification

I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each ...
7
votes
1answer
115 views

Corpus of human-scored machine translations?

Most of the parallel corpora (Opus, EuroParl, OpenSubtitles) have only human translations. src_txt | human_trans_of_src_txt_into_trg_lng (We assume all the translations are good.) Is there a corpus ...
4
votes
1answer
81 views

Access to anonymous personal financial portfolio data

Is there a corpus about personal financial portfolios? I.e. data which is anonymous user data and shows financial portfolios for these users.
6
votes
1answer
59 views

Corpus of documents with important sentences marked

I'm looking to create a sentence extraction program, so a program that aims to get the most important sentences from a body of text. The first step for me is to try to evaluate what characteristics ...
5
votes
2answers
352 views

Text data sources in German, primarily news articles

Can anyone recommend good sources for getting archived news texts in German language? Good source would satisfy the following conditions: recent in date (for example 2005-2015) variety of topics (...
6
votes
2answers
104 views

Looking for modern english texts in public domain / CC-BY-SA

Is there a library of public domain / CC-BY-SA english texts, written later than 1990? I need it to build a corpora with links to full texts, which isn't possible with copyrighted material. At the ...
2
votes
1answer
147 views

Dialogs with labeled slot(s)-value(s)

I am looking for data sets containing dialogues labeled with slot-values. I am open as to the definition of a dialogue, e.g. message board posts are good too. Definition of slot-values: A dialog is ...
5
votes
2answers
313 views

Open text document corpus for information retrieval evaluation

INTRODUCTION: document collections (corpora) for evaluation of information retrieval (search engine) systems are pretty often behind a paywall. A notorious example is the TREC conference (http://trec....
4
votes
0answers
39 views

Novels with chapters annotated POV

I'm looking for one or more novels, that multiple characters points of view. (I don't care if it is 3rd person or 1st person), with each chapter (or paragraph) annotated as to which character's POV ...
3
votes
1answer
181 views

source of character-level ngrams that include spaces

I'm looking for a source of character-level ngrams and counts that include spaces. I've found character-level ngram datasets, but they don't include spaces. Ideally, the dataset will include up to 8-...
4
votes
1answer
53 views

Novel Blurb Corpus

I am looking for a corpus of fiction book blurbs. With at least 10,000 entries. Preferably with a lot more. I do not care if it is a single publisher (etc) or many. It should be annotated with ...
3
votes
2answers
718 views

SQL queries dataset

I am looking for a dataset containing natural language sentences (English preferred) and their corresponding SQL queries. E.g. "What are the names of the employees who live in Los Angeles" could be ...
5
votes
4answers
95 views

Tagged addresses

Does anyone know of a corpus of US addresses where the parts of the addresses have been tagged. Something like <streetNumber>123</streetNumber> <streetDirection>E.</...
7
votes
2answers
251 views

French equivalent of the brown corpus

The brown corpus is a collection of text where each element is already gramatically tagged. It contains about one million words and is often used to train statistical PoS taggers. For the french ...
2
votes
0answers
66 views

Dataset of regexps with corresponding sentences

I'm looking for datasets that contain a list of regexps where each regexp is associated with at least one sentence written in English that explains what the regexp does. Examples: "All 3-letter words ...
11
votes
7answers
3k views

Large French dataset for NLP (not formal, rather discussions/reviews)

What am I looking for ? I'm looking for a big dataset (at least a few GB) containing French text. This french text should not have too much quality (such as literature or encyclopedias). Why this ...
8
votes
3answers
920 views

Set of handwritten, labeled characters

I'm looking for a large labelled data set, of characters that I can use to contrast recognising Letters, with recognising Numbers. Something very similar to MNIST, but with letters. Language doesn't ...
13
votes
1answer
2k views

Speech audio databases with phonemes labelled

Can anyone point me to opensource databases of speech audio with phonemes labelled, .wav or the like + label files, e.g. # time, phoneme 0.31200 ao 0.41200 th 0.54200 er ... I'd prefer English, ...
9
votes
1answer
589 views

Text message corpus for American english

I am doing a research project to identify some patterns in how people interact with each other using text messages. The scope of the project is confined to American users. Can someone suggest a good ...
11
votes
2answers
2k views

American English SMS Text Message Corpora

I'm seeking corpora of American English SMS (<=160 characters) text messages. What corpora are available?