Skip to main content
Share Your Experience: Take the 2024 Developer Survey
6 votes
Accepted

Is there any picture corpus?

You may be able to use the International Picture Naming Project (IPNP), but I think the total pictures are way less than 10k. Here's a link to the query tool. You may get lucky searching through "...
philshem's user avatar
  • 17.6k
5 votes

Swedish Noun Gender List

Probably the Swedish Academian Wordlist (SAOL) is the most authoritative source in this field. There exists SAOL for Android, the .obb expansion file is a regular SQLite database. This list was ...
Stanislav Kralin's user avatar
3 votes

Is there any picture corpus?

Wikidata has linked various concepts and lexemes to images. For lexemes and their senses there are not that many images yet. For concepts there are many, but many are proper nouns or complex concepts, ...
Finn Årup Nielsen's user avatar
3 votes

American English SMS Text Message Corpora

I recently made a corpus available of text messages from Spanish-dominant bilingual young adults in New York City (Bilingual Youth Texts Corpus). It is at www.byts.commons.gc.cuny.edu
MichelleMc's user avatar
3 votes
Accepted

Large English text corpus

Project Gutenberg offers 57.000 free books, available in different formats. Among them, utf-8 encoded plain text with minimal formatting. The NLTK comes with access to a range of corpora. Among them, ...
Lutz Büch's user avatar
3 votes

Audio data set of (telephone) conversations and their transcriptions

Most telephone recording datasets are privately owned by parties who have access to such data like for example call-centers and tech-support companies etc. This data is subject to various privacy laws ...
SuperKogito's user avatar
3 votes

Sentence segmentation dataset

The Pragmatic Segmenter segmentation library contains sentences for the common edge cases in English as well as some other languages. The README also includes links to a lot of research and other ...
diasks2's user avatar
  • 131
3 votes
Accepted

Corpus of documents with important sentences marked

I actually ended up paying folks on MechanicalTurk to label questions as important/unimportant from a couple of news articles I downloaded. There are 410 sentences total, which I have on my github ...
John Madden's user avatar
2 votes

Text data sources in German, primarily news articles

We release large corpora of German news for the annual machine translation shared task organised by the WMT conference. Unfortunately it is shuffled (so you cannot reconstruct the original articles) ...
bazril's user avatar
  • 21
2 votes

Corpus of tagged text (English newspapers or any tagged text)

You may also want to check the American National Corpus and well as the holdings of the Linguistic Data Consortium.
Brandon Loudermilk's user avatar
2 votes

Swedish Noun Gender List

dict.cc has downloadable dictionaries https://www1.dict.cc/translation_file_request.php?l=e You can choose either English (EN) to Swedish (SV) or from German (DE) to SV, and the opposite direction. ...
philshem's user avatar
  • 17.6k
2 votes

Hate speech text data

The Internet Archive's Wayback Machine has indexed the text of Nazi and white supremacist message boards, such as Stormfront and VNN. You can download the WARC's (snapshot compilations of the ...
Asparagirl's user avatar
2 votes

In Google Books Ngram datasets, what are the meanings of '._.' (dot-underscore-dot) and '_._' (underscore-dot-underscore)?

Ah, it is answered in the second paper after all, as explained in this answer over on StackOverflow. The tags such as _NOUN_ etc. can either stand alone (_NOUN_) or can be appended to a word (...
linguisticturn's user avatar
2 votes

Actual text and corrected text? Or autocorrect data?

OK, this one is a stretch... but instead of comparing text with errors to the corrected text, you can compare original text against incorrect transcriptions. Reuters Transcribed Subset and data set ...
philshem's user avatar
  • 17.6k
1 vote

Is there a corpus of "nice", inoffensive English words?

This positive words text file posted by a user on GitHub. It contains 2000+ positive English's words.
Shivam Maurya 1729's user avatar
1 vote
Accepted

Speech intelligiblity tests such as rhyme tests

A good metric for speech intelligibility is the Short-Time Objective Intelligibility (STOI). Since your targets are hearing aids, if you manage to have a some sample audios recorded at the aid output, ...
SuperKogito's user avatar
1 vote

Financial Corpus

Reuters Financial Dataset as a structured DataFrame Reuters Financial Dataset is a large collection of Financial News Article scraped from Reuters website. Originally used for the paper Using ...
Orophile's user avatar
  • 1,761
1 vote

Corpus suggestion for financial domain

The US Securities and Exchange Commission requires larger publicly traded companies to file quarterly and annual financial statements in XMRL. These can be accessed from the agency's EDGAR site. In ...
Richard Careaga's user avatar
1 vote

Large English text corpus

I have posted a collection of several text datasets related to news here. They contain 50+ mb of text each in an easily parsable format. https://www.kaggle.com/therohk/datasets
Rohit's user avatar
  • 141
1 vote

Books with multiple main characters, one per chapter

This may be a question that the human brain answers really quick, and a computational approach way longer to design. Why not "crowdsource" by posting on some literary/book forums (e.g. https:...
philshem's user avatar
  • 17.6k
1 vote

Download OpenOffice/Hunspell dictionaries (.oxt / .dic+.aff) for many languages at once?

This site has the dictionaries. https://github.com/LibreOffice/dictionaries
djnavas's user avatar
  • 119
1 vote

Corpus of words divided by language

Consider wikidata. Wikidata is a site that linked all different articles from different wikipedia together, as such article name of these wikipedia articles would form a list of word for each language ...
user930067's user avatar
1 vote

Human name country wise corpus

Since 2014 the Dutch social insurance bank publishes lists of most popular boys and girls names: for boys: http://svbvod.download.kpnstreaming.nl/kindernamen-2016/jongens-populair-2016.pdf for girls: ...
Swier's user avatar
  • 1,453
1 vote

Human name country wise corpus

UK first names are available from birth registration data from the ONS here (annual files are available and there is also data going back a lot of years). Trends and visualisation of some of this data ...
matt_black's user avatar
1 vote

Database of Press Releases

Most publicly traded companies (and some private ones) will keep a list of recent press releases on their website. Sometimes Yahoo Finance, when "covering" a publicly traded company, will compile a ...
Tom Au's user avatar
  • 541
1 vote

Corpus of tagged text (English newspapers or any tagged text)

Get the CoNLL-2003: data from this repo The CoNLL-2003 shared task data files contain four columns separated by a single space. Each word has been put on a separate line and there is an empty line ...
Annu Sachan's user avatar
1 vote

Corpus of human-scored machine translations?

Such a dataset was used to create the BLEU metric, but I do not know if it is available. Maybe you could try asking the authors of the paper that introduced it if they could share their data? ...
pintoch's user avatar
  • 321
1 vote

Dialogs with labeled slot(s)-value(s)

The Dialog State Tracking Challenge 4 (DSTC 4): consists of 35 dialog sessions on touristic information for Singapore, collected from Skype calls between three tour guides and 35 tourists. Each dialog ...
Franck Dernoncourt's user avatar
1 vote

Set of handwritten, labeled characters

Do check an image dataset at KAGGLE it contains A-Z handwritten 370000+ images
Sachin Patel's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible