Questions tagged [corpora]

A database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.

Filter by
Sorted by
Tagged with
2 votes
1 answer
62 views

Corpus of ChatGPT 4.0 responses

Does anyone have a repository of ChatGPT 4.0 responses (along with their prompts)? I want English non-fiction (especially technical writing), which could include subjects like programming, science, or ...
Laurel's user avatar
  • 127
1 vote
2 answers
59 views

Word frequency dataset across many languages?

I know there is the Wikipedia word frequency lists, but you would have to write a parser for each of those, which is a lot of work, to get the data. I found these two repos: IlyaSemenov/wikipedia-...
Lance's user avatar
  • 387
3 votes
2 answers
54 views

Is there a corpus of "nice", inoffensive English words?

I'm looking for an English language corpus that doesn't include profanity or "unpleasant" words to generate easy to remember passwords and other strings for users. Does anybody know of such ...
kontextify's user avatar
2 votes
1 answer
15 views

In Google Books Ngram datasets, what are the meanings of '._.' (dot-underscore-dot) and '_._' (underscore-dot-underscore)?

Consider the Version 3 dataset for 2-grams labeled 'Brunswick_NOUN Memorial_NOUN' (its actual name is '2-00109-of-00589.gz'). Compare the following three entries: Buitenhuis .      1859,2,2           ...
linguisticturn's user avatar
1 vote
0 answers
27 views

Speech intelligiblity tests such as rhyme tests

Can anyoe point me to .wav files or the like for speech intelligiblity tests, e.g. Rhyme tests "not, tot, got, pot, hot, lot" ? What I want are reproducible tests for various hearing aids ...
denis's user avatar
  • 456
1 vote
0 answers
12 views

Seeking database, digital corpus, or text archive of Zen Buddhist texts

Particularly interested in Sutras and Koans in English (open domain).
Brandon Loudermilk's user avatar
1 vote
1 answer
63 views

Is there a test dataset of research papers in computer science with relevance judgments

Is there a freely available test dataset of research papers in computer science field? I need to evaluate my text retrieval system which uses a computer science domain ontology called CSO, so the ...
Nina's user avatar
  • 111
2 votes
1 answer
343 views

Financial Corpus

I'll be delving into text mining applications for my master thesis and I need data for it. Ideally, I would need a corpora of texts/news articles from some single (or multiple) credible and ...
Tomi Sargiotto's user avatar
0 votes
1 answer
87 views

Corpus suggestion for financial domain

I am looking for a financial corpus or any form of publicly available financial texts which is replete with technical terms and acronyms. Any suggestion is appreciated
FakhroSadat Mousavi's user avatar
2 votes
2 answers
145 views

Large English text corpus

I am interested in studying a few specific questions on entropy of different properties of English text. For instance, what is the entropy of capital letters in English? This might tell you something ...
Engineero's user avatar
  • 125
1 vote
0 answers
27 views

What are methods to create open text data from copyrighted texts?

Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
user avatar
5 votes
1 answer
2k views

Audio data set of (telephone) conversations and their transcriptions

I need records of telephone conversations (or simple conversations between two persons) with their transcripts. I need English and French datasets.
K.cyrine's user avatar
1 vote
1 answer
61 views

Books with multiple main characters, one per chapter

I am looking for public domain, or creative commons fiction, with: multiple main characters, written in 3rd person Each chapter focuses on a particular character Modern examples of such books ...
Frames Catherine White's user avatar
1 vote
0 answers
50 views

Licensing of machine learning tool trained on open data

The broad question is: how are the licensing terms for a classifier that is trained with open annotated (or manually annotated) data? I am trying to train a dependency parser for German text with ...
Miluleh's user avatar
  • 11
3 votes
2 answers
578 views

Swedish Noun Gender List

I'm looking for a lexical resource on Swedish noun genders. In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
Nordlöw's user avatar
  • 261
2 votes
0 answers
123 views

Spanish POS (part of speech) tagged dataset

Where can I get a spanish POS tagged dataset with universal tags free to download?
kendall's user avatar
  • 21
4 votes
1 answer
347 views

Hate speech text data

Where can I find a collection of hate speech articles and texts? I have tried using hate speech and offensive language from github but I am looking for long pieces of text (>~160 characters).
mohamed emad's user avatar
3 votes
1 answer
729 views

Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?

I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org is quite cumbersome at that scale. Is there a command ...
Adam Bittlingmayer's user avatar
3 votes
2 answers
80 views

Corpus of words divided by language

I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language. I don't really care for translations, ...
ClonedOne's user avatar
  • 133
8 votes
2 answers
640 views

Human name country wise corpus

Where I can get the human name corpus country-wise, so that I can use it for training my neural network for detecting the human names out of the string. Does anyone have idea of this?
Jaffer Wilson's user avatar
3 votes
1 answer
73 views

Database of Press Releases

Is there a corpus of business/corporate press releases? I'm looking for a collection of releases like these, but from any corporation whatsoever: http://www.businesswire.com/news/home/20161209005816/...
szxk's user avatar
  • 810
5 votes
1 answer
280 views

Actual text and corrected text? Or autocorrect data?

I seek essentially a parallel corpora, which has the dirty original user-generated input on the one side, and a cleaned up version on the other side - with corrected spelling, casing, punctuation and ...
Adam Bittlingmayer's user avatar
4 votes
3 answers
591 views

Is there any picture corpus?

By "picture corpus" I mean the picture collection each illustrating one word in the usually used vocabulary, such as words in Google 10000 English I found on GitHub. For instance: vocabulary ...
Lerner Zhang's user avatar
3 votes
1 answer
273 views

Sentence segmentation dataset

I am looking for datasets that contain texts as well as notations for sentence boundaries. I plan to use it to assess a sentence segmentation system. I am mostly interested in texts in English at the ...
Franck Dernoncourt's user avatar
10 votes
4 answers
5k views

Corpus of tagged text (English newspapers or any tagged text)

I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks. NOTE: By ...
user10492's user avatar
  • 101
2 votes
1 answer
241 views

Big data set for Document Classification

I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each ...
Eyad's user avatar
  • 21
7 votes
1 answer
143 views

Corpus of human-scored machine translations?

Most of the parallel corpora (Opus, EuroParl, OpenSubtitles) have only human translations. src_txt | human_trans_of_src_txt_into_trg_lng (We assume all the translations are good.) Is there a corpus ...
Adam Bittlingmayer's user avatar
4 votes
1 answer
96 views

Access to anonymous personal financial portfolio data

Is there a corpus about personal financial portfolios? I.e. data which is anonymous user data and shows financial portfolios for these users.
Malhar Gupta's user avatar
7 votes
1 answer
94 views

Corpus of documents with important sentences marked

I'm looking to create a sentence extraction program, so a program that aims to get the most important sentences from a body of text. The first step for me is to try to evaluate what characteristics ...
John Madden's user avatar
4 votes
2 answers
514 views

Text data sources in German, primarily news articles

Can anyone recommend good sources for getting archived news texts in German language? Good source would satisfy the following conditions: recent in date (for example 2005-2015) variety of topics (...
user avatar
5 votes
2 answers
142 views

Looking for modern english texts in public domain / CC-BY-SA

Is there a library of public domain / CC-BY-SA english texts, written later than 1990? I need it to build a corpora with links to full texts, which isn't possible with copyrighted material. At the ...
Denis Kulagin's user avatar
2 votes
1 answer
224 views

Dialogs with labeled slot(s)-value(s)

I am looking for data sets containing dialogues labeled with slot-values. I am open as to the definition of a dialogue, e.g. message board posts are good too. Definition of slot-values: A dialog is ...
Franck Dernoncourt's user avatar
5 votes
2 answers
558 views

Open text document corpus for information retrieval evaluation

INTRODUCTION: document collections (corpora) for evaluation of information retrieval (search engine) systems are pretty often behind a paywall. A notorious example is the TREC conference (http://trec....
Jason Doeson's user avatar
4 votes
0 answers
45 views

Novels with chapters annotated POV

I'm looking for one or more novels, that multiple characters points of view. (I don't care if it is 3rd person or 1st person), with each chapter (or paragraph) annotated as to which character's POV ...
Frames Catherine White's user avatar
3 votes
1 answer
314 views

source of character-level ngrams that include spaces

I'm looking for a source of character-level ngrams and counts that include spaces. I've found character-level ngram datasets, but they don't include spaces. Ideally, the dataset will include up to 8-...
engram's user avatar
  • 31
4 votes
1 answer
143 views

Novel Blurb Corpus

I am looking for a corpus of fiction book blurbs. With at least 10,000 entries. Preferably with a lot more. I do not care if it is a single publisher (etc) or many. It should be annotated with ...
Frames Catherine White's user avatar
3 votes
2 answers
971 views

SQL queries dataset

I am looking for a dataset containing natural language sentences (English preferred) and their corresponding SQL queries. E.g. "What are the names of the employees who live in Los Angeles" could be ...
Franck Dernoncourt's user avatar
5 votes
4 answers
122 views

Tagged addresses

Does anyone know of a corpus of US addresses where the parts of the addresses have been tagged. Something like <streetNumber>123</streetNumber> <streetDirection>E.</...
fgregg's user avatar
  • 5,108
7 votes
2 answers
458 views

French equivalent of the brown corpus

The brown corpus is a collection of text where each element is already gramatically tagged. It contains about one million words and is often used to train statistical PoS taggers. For the french ...
merours's user avatar
  • 413
2 votes
0 answers
72 views

Dataset of regexps with corresponding sentences

I'm looking for datasets that contain a list of regexps where each regexp is associated with at least one sentence written in English that explains what the regexp does. Examples: "All 3-letter words ...
Franck Dernoncourt's user avatar
11 votes
7 answers
7k views

Large French dataset for NLP (not formal, rather discussions/reviews)

What am I looking for ? I'm looking for a big dataset (at least a few GB) containing French text. This french text should not have too much quality (such as literature or encyclopedias). Why this ...
merours's user avatar
  • 413
8 votes
3 answers
1k views

Set of handwritten, labeled characters

I'm looking for a large labelled data set, of characters that I can use to contrast recognising Letters, with recognising Numbers. Something very similar to MNIST, but with letters. Language doesn't ...
Frames Catherine White's user avatar
14 votes
2 answers
2k views

Speech audio databases with phonemes labelled

Can anyone point me to opensource databases of speech audio with phonemes labelled, .wav or the like + label files, e.g. # time, phoneme 0.31200 ao 0.41200 th 0.54200 er ... I'd prefer English, ...
denis's user avatar
  • 456
9 votes
1 answer
684 views

Text message corpus for American english

I am doing a research project to identify some patterns in how people interact with each other using text messages. The scope of the project is confined to American users. Can someone suggest a good ...
Darth.Vader's user avatar
10 votes
2 answers
4k views

American English SMS Text Message Corpora

I'm seeking corpora of American English SMS (<=160 characters) text messages. What corpora are available?
Dan's user avatar
  • 520