16 votes
Accepted

Large list of quotes

Have you seen WikiQuote by the Wikimedia Foundation? There is an API endpoint at http://en.wikiquote.org/w/api.php which uses the standard MediaWiki API for there are API clients in many different ...
11 votes

Database world cities with their names in all languages of the world

Wikidata has a category called "city" which will give you a list of world cities https://www.wikidata.org/wiki/Q515 You can use SPARQL to query Wikidata https://www.mediawiki.org/wiki/...
  • 17.3k
10 votes
Accepted

Dataset of sentences translated into many languages

Tatoeba.org is exactly that: http://tatoeba.org Their data is human-edited and used by dozens of products/websites including electronic dictionaries, so it is of reasonable quality. It has 471468 ...
10 votes

Is there a free list of English word phonetics?

The cmudict provides phonetic spellings of a sizable number of American English words. The CELEX database is a similar project; you can select which data you want and download wordlists at WebCelex. ...
  • 799
9 votes
Accepted

Speech audio files dataset with language labels

You can use the Tatoeba website which has full sentences in text and audio as downloads. Sentences with audio Download http://downloads.tatoeba.org/exports/sentences_with_audio.tar.bz2 Fields and ...
  • 17.3k
9 votes

Database world cities with their names in all languages of the world

I guess the Geonames project is the closest answer to your question. You can download the data directly on the website or you can also find it here and download it/use the api. Cheers
  • 1,157
8 votes
Accepted

List of abbreviations and acronyms

I found a github repository that scrapes the wiki list of roughly 6000 acronyms https://github.com/krishnakt031990/Crawl-Wiki-For-Acronyms raw acronyms file and then I wrote this python script to ...
  • 17.3k
8 votes

Speech audio files dataset with language labels

I suggest looking at the SpokenLanguages2 dataset, which is very extensive, containing a bit more than one hour of speech for each of the 172 languages: https://community.topcoder.com/longcontest/?...
  • 81
7 votes

Speech audio files dataset with language labels

UCLA Phonetics Lab Archive offers many audio files of spoken word in many languages, although you'll have to dig through and mix-match to your liking: http://archive.phonetics.ucla.edu/ IPA Handbook ...
  • 11.8k
7 votes

Speech audio files dataset with language labels

Maybe you can do something with https://rhinospike.com/language/ It's a bit like Tatoeba mentioned above edit : Take also a look at librivox (public domain audiobooks). It comes with an API, then you ...
  • 131
7 votes
Accepted

Audio Datasets Featuring Different Speakers Saying the Same Sentence (English)?

The speech accent archive: uniformly presents a large set of speech samples from a variety of language backgrounds. As for English, the following subset contains the sentences: Please call ...
6 votes

Dataset of languages and where they are spoken (sub-national)

The CIA Word Factbook has a field listing for languages spoken per country and percentage of population that speaks it. https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html ...
6 votes

Database of English words pronunciation

Take a look at Project SHTOOKA
6 votes

Is a spoken digit dataset available?

https://github.com/Jakobovski/free-spoken-digit-dataset is a free spoken digit dataset (FSDD). As an added bonus it comes with a few useful python utility functions. I created this dataset because I ...
6 votes

Speech audio files dataset with language labels

I've found something else. It is called the wide language index. See on github. It consists in a listing of radio podcasts. It's pretty impressive in the numbers of languages covered. Note : It was ...
  • 131
6 votes

Database world cities with their names in all languages of the world

As a variation of above, if you want all the labels in all the languages, you can get it from Wikidata like so: SELECT DISTINCT ?city ?country (lang(?label) AS ?language) ?label WHERE { ?city ...
6 votes
Accepted

How to collect hand-writting data

The first step is splitting the image into character arrays. To do that, check out the answers in this question: Separate image of text into component character images. In particular, the ImageMagick ...
  • 17.3k
6 votes
Accepted

Is there any picture corpus?

You may be able to use the International Picture Naming Project (IPNP), but I think the total pictures are way less than 10k. Here's a link to the query tool. You may get lucky searching through "...
  • 17.3k
6 votes

Where can I find a list of latin words?

You can get 8600 Latin words from Latin wikitionary or 879.898 Latin words from English wiktionary using PetScan. You can change output to csv or tsv in the "Output" tab.
  • 381
5 votes

Phones dataset for speech recognition (not telephone number)

You can find US and British english language phones at university linguistic departments such as Berkeley, Ca. Though, what is useful for speech recognition are the features in statistical acoustic ...
  • 131
5 votes

Data set of news articles and scientific journals

As far as I know, one of the best data sources for NLP has been the ENRON emails. While this is not a "news" source, it would certainly provide some raw data for you to play around with. Another ...
5 votes

List of abbreviations and acronyms

There doesn't seem to be a proper open acronym project, unfortunately. Data source The best solution at the moment seems to be Wiktionary, which has a category called Initialism (a synonym for ...
  • 17.3k
5 votes

Are there opendatasets of English terms describing nationalities, ethnicities?

WordNet is a (free) semantic database of the English language. You could query for example the complete hyponym tree of the term "inhabitant". Depending on your application, it could be a disadvantage ...
  • 406
5 votes

Where can I find a part-of-speech corpus free for commercial use?

After doing some searching it seems that the Moby Project is in the public domain, and they have a POS corpus. However, it's simply a dictionary so it doesn't help with words that have multiple POS. ...
5 votes

Single-word Translations in All Languages

I'm not aware of an available dataset in the requested form, but it should be possible to obtain something similar by parsing data from wiktionary. The latest dump can be obtained at https://dumps....
5 votes

German news/text data set

I think a good source would be the COSMAS II-Korpora from the Institut für Deutsche Sprache. It seems there is free registration and downloads for non-commercial use.
  • 17.3k
5 votes

ASCII Character Frequency Analysis

You can pretty easily create character (or word) frequency data based on a few lines of code. Here is python 2.7 code that collects counts of each character # -*- coding: utf-8 -*- import string ...
  • 17.3k
5 votes

Where can I find a list of latin words?

I haven't found any ready lists with just Latin words in txt format, but you can parse these lists to extract the Latin words from them: Dickinson College: LATIN CORE VOCABULARY. Terms of use: "(...)...
  • 229
5 votes

Swedish Noun Gender List

Probably the Swedish Academian Wordlist (SAOL) is the most authoritative source in this field. There exists SAOL for Android, the .obb expansion file is a regular SQLite database. This list was ...

Only top scored, non community-wiki answers of a minimum length are eligible