Questions tagged [corpora]
A database of text or speech, possibly annotated with language-specific explanatory information. Used for testing statistical hypothesis and constructing quantitative models.
45 questions
3
votes
1
answer
82
views
Corpus of ChatGPT 4.0 responses
Does anyone have a repository of ChatGPT 4.0 responses (along with their prompts)?
I want English non-fiction (especially technical writing), which could include subjects like programming, science, or ...
- 153
2
votes
2
answers
215
views
Word frequency dataset across many languages?
I know there is the Wikipedia word frequency lists, but you would have to write a parser for each of those, which is a lot of work, to get the data.
I found these two repos:
IlyaSemenov/wikipedia-...
- 439
3
votes
2
answers
59
views
Is there a corpus of "nice", inoffensive English words?
I'm looking for an English language corpus that doesn't include profanity or "unpleasant" words to generate easy to remember passwords and other strings for users. Does anybody know of such ...
- 149
2
votes
1
answer
20
views
In Google Books Ngram datasets, what are the meanings of '._.' (dot-underscore-dot) and '_._' (underscore-dot-underscore)?
Consider the Version 3 dataset for 2-grams labeled 'Brunswick_NOUN Memorial_NOUN' (its actual name is '2-00109-of-00589.gz'). Compare the following three entries:
Buitenhuis . 1859,2,2 ...
- 141
3
votes
1
answer
45
views
Speech intelligiblity tests such as rhyme tests
Can anyoe point me to .wav files or the like for
speech intelligiblity tests, e.g.
Rhyme tests
"not, tot, got, pot, hot, lot" ?
What I want are reproducible tests for various hearing aids ...
- 498
1
vote
0
answers
13
views
Seeking database, digital corpus, or text archive of Zen Buddhist texts
Particularly interested in Sutras and Koans in English (open domain).
1
vote
1
answer
67
views
Is there a test dataset of research papers in computer science with relevance judgments
Is there a freely available test dataset of research papers in computer science field? I need to evaluate my text retrieval system which uses a computer science domain ontology called CSO, so the ...
- 111
2
votes
1
answer
412
views
Financial Corpus
I'll be delving into text mining applications for my master thesis and I need data for it. Ideally, I would need a corpora of texts/news articles from some single (or multiple) credible and ...
0
votes
1
answer
91
views
Corpus suggestion for financial domain
I am looking for a financial corpus or any form of publicly available financial texts which is replete with technical terms and acronyms.
Any suggestion is appreciated
2
votes
2
answers
156
views
Large English text corpus
I am interested in studying a few specific questions on entropy of different properties of English text. For instance, what is the entropy of capital letters in English? This might tell you something ...
- 125
1
vote
0
answers
27
views
What are methods to create open text data from copyrighted texts?
Language data are either texts or spoken utterances. By default, any text is copyrighted. We have Open Data for relatively recent time periods, where text were written intentionally with an open ...
user6083
5
votes
1
answer
3k
views
Audio data set of (telephone) conversations and their transcriptions
I need records of telephone conversations (or simple conversations between two persons) with their transcripts. I need English and French datasets.
- 51
1
vote
1
answer
75
views
Books with multiple main characters, one per chapter
I am looking for public domain, or creative commons fiction,
with:
multiple main characters,
written in 3rd person
Each chapter focuses on a particular character
Modern examples of such books ...
1
vote
0
answers
50
views
Licensing of machine learning tool trained on open data
The broad question is: how are the licensing terms for a classifier that is
trained with open annotated (or manually annotated) data?
I am trying to train a dependency parser for German text with ...
- 11
3
votes
2
answers
636
views
Swedish Noun Gender List
I'm looking for a lexical resource on Swedish noun genders.
In my case which nouns that have gender neuter or not, that is if the indefinite article is en or ett.
- 261
2
votes
0
answers
124
views
Spanish POS (part of speech) tagged dataset
Where can I get a spanish POS tagged dataset with universal tags free to download?
- 21
4
votes
1
answer
348
views
Hate speech text data
Where can I find a collection of hate speech articles and texts? I have tried using hate speech and offensive language from github but I am looking for long pieces of text (>~160 characters).
- 41
3
votes
1
answer
1k
views
Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?
I would like dictionaries for the top ~100 languages or so. Manually searching and evaluating the results at https://extensions.openoffice.org is quite cumbersome at that scale.
Is there a command ...
3
votes
2
answers
80
views
Corpus of words divided by language
I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language.
I don't really care for translations, ...
- 133
8
votes
2
answers
651
views
Human name country wise corpus
Where I can get the human name corpus country-wise, so that I can use it for training my neural network for detecting the human names out of the string.
Does anyone have idea of this?
- 221
3
votes
1
answer
77
views
Database of Press Releases
Is there a corpus of business/corporate press releases? I'm looking for a collection of releases like these, but from any corporation whatsoever: http://www.businesswire.com/news/home/20161209005816/...
- 810
5
votes
1
answer
280
views
Actual text and corrected text? Or autocorrect data?
I seek essentially a parallel corpora, which has the dirty original user-generated input on the one side, and a cleaned up version on the other side - with corrected spelling, casing, punctuation and ...
4
votes
3
answers
606
views
Is there any picture corpus?
By "picture corpus" I mean the picture collection each illustrating one word in the usually used vocabulary, such as words in Google 10000 English I found on GitHub.
For instance: vocabulary
...
- 265
3
votes
1
answer
274
views
Sentence segmentation dataset
I am looking for datasets that contain texts as well as notations for sentence boundaries. I plan to use it to assess a sentence segmentation system. I am mostly interested in texts in English at the ...
- 7,815
10
votes
4
answers
5k
views
Corpus of tagged text (English newspapers or any tagged text)
I'm developing a system to extract tags from text (English) and currently I have no dataset to test the system and evaluate, could someone point me to a source (preferably a free one) thanks.
NOTE:
By ...
- 101
3
votes
1
answer
250
views
Big data set for Document Classification
I'm looking for big data set which is suitable to be used for document classification task. The data set which I'm looking for should composed of the frequency of the words which exist in each ...
- 31
7
votes
1
answer
143
views
Corpus of human-scored machine translations?
Most of the parallel corpora (Opus, EuroParl, OpenSubtitles) have only human translations.
src_txt | human_trans_of_src_txt_into_trg_lng
(We assume all the translations are good.)
Is there a corpus ...
4
votes
1
answer
97
views
Access to anonymous personal financial portfolio data
Is there a corpus about personal financial portfolios? I.e. data which is anonymous user data and shows financial portfolios for these users.
- 51
7
votes
1
answer
96
views
Corpus of documents with important sentences marked
I'm looking to create a sentence extraction program, so a program that aims to get the most important sentences from a body of text. The first step for me is to try to evaluate what characteristics ...
- 201
4
votes
2
answers
515
views
Text data sources in German, primarily news articles
Can anyone recommend good sources for getting archived news texts in German language?
Good source would satisfy the following conditions:
recent in date (for example 2005-2015)
variety of topics (...
enedene
5
votes
2
answers
146
views
Looking for modern english texts in public domain / CC-BY-SA
Is there a library of public domain / CC-BY-SA english texts, written later than 1990? I need it to build a corpora with links to full texts, which isn't possible with copyrighted material. At the ...
- 291
2
votes
1
answer
226
views
Dialogs with labeled slot(s)-value(s)
I am looking for data sets containing dialogues labeled with slot-values. I am open as to the definition of a dialogue, e.g. message board posts are good too.
Definition of slot-values:
A dialog is ...
- 7,815
5
votes
2
answers
564
views
Open text document corpus for information retrieval evaluation
INTRODUCTION: document collections (corpora) for evaluation of information retrieval (search engine) systems are pretty often behind a paywall. A notorious example is the TREC conference (http://trec....
- 51
4
votes
0
answers
45
views
Novels with chapters annotated POV
I'm looking for one or more novels, that multiple characters points of view.
(I don't care if it is 3rd person or 1st person),
with each chapter (or paragraph) annotated as to which character's POV ...
4
votes
1
answer
336
views
source of character-level ngrams that include spaces
I'm looking for a source of character-level ngrams and counts that include spaces. I've found character-level ngram datasets, but they don't include spaces. Ideally, the dataset will include up to 8-...
- 41
4
votes
1
answer
147
views
Novel Blurb Corpus
I am looking for a corpus of fiction book blurbs.
With at least 10,000 entries.
Preferably with a lot more.
I do not care if it is a single publisher (etc) or many.
It should be annotated with ...
3
votes
2
answers
984
views
SQL queries dataset
I am looking for a dataset containing natural language sentences (English preferred) and their corresponding SQL queries.
E.g. "What are the names of the employees who live in Los Angeles" could be ...
- 7,815
5
votes
4
answers
122
views
Tagged addresses
Does anyone know of a corpus of US addresses where the parts of the addresses have been tagged.
Something like
<streetNumber>123</streetNumber>
<streetDirection>E.</...
- 5,108
7
votes
2
answers
466
views
French equivalent of the brown corpus
The brown corpus is a collection of text where each element is already gramatically tagged. It contains about one million words and is often used to train statistical PoS taggers.
For the french ...
- 413
2
votes
0
answers
74
views
Dataset of regexps with corresponding sentences
I'm looking for datasets that contain a list of regexps where each regexp is associated with at least one sentence written in English that explains what the regexp does. Examples:
"All 3-letter words ...
- 7,815
11
votes
7
answers
7k
views
Large French dataset for NLP (not formal, rather discussions/reviews)
What am I looking for ?
I'm looking for a big dataset (at least a few GB) containing French text. This french text should not have too much quality (such as literature or encyclopedias).
Why this ...
- 413
8
votes
3
answers
1k
views
Set of handwritten, labeled characters
I'm looking for a large labelled data set, of characters
that I can use to contrast recognising Letters,
with recognising Numbers.
Something very similar to MNIST,
but with letters.
Language doesn't ...
14
votes
2
answers
2k
views
Speech audio databases with phonemes labelled
Can anyone point me to opensource databases of speech audio with phonemes labelled,
.wav or the like + label files, e.g.
# time, phoneme
0.31200 ao
0.41200 th
0.54200 er
...
I'd prefer English, ...
- 498
9
votes
1
answer
686
views
Text message corpus for American english
I am doing a research project to identify some patterns in how people interact with each other using text messages. The scope of the project is confined to American users. Can someone suggest a good ...
- 191
10
votes
2
answers
4k
views
American English SMS Text Message Corpora
I'm seeking corpora of American English SMS (<=160 characters) text messages. What corpora are available?
- 520