16 votes
Accepted

Large list of quotes

Have you seen WikiQuote by the Wikimedia Foundation? There is an API endpoint at http://en.wikiquote.org/w/api.php which uses the standard MediaWiki API for there are API clients in many different ...
user avatar
12 votes

List of English words

You can download an Aspell Dictionary, then convert it to simple list of words: aspell -d en dump master | aspell -l en expand > my.dict A few other dictionaries.
user avatar
10 votes

Is there a free list of English word phonetics?

The cmudict provides phonetic spellings of a sizable number of American English words. The CELEX database is a similar project; you can select which data you want and download wordlists at WebCelex. ...
user avatar
  • 789
10 votes

Database world cities with their names in all languages of the world

Wikidata has a category called "city" which will give you a list of world cities https://www.wikidata.org/wiki/Q515 You can use SPARQL to query Wikidata https://www.mediawiki.org/wiki/...
user avatar
  • 17.1k
9 votes

List of English words

Use wordnet, it's an academic project but a de facto for various purposes including the one you asked for. There are also higher semantics modeled that you can use for various functions. It's also ...
user avatar
9 votes
Accepted

Dataset of sentences translated into many languages

Tatoeba.org is exactly that: http://tatoeba.org Their data is human-edited and used by dozens of products/websites including electronic dictionaries, so it is of reasonable quality. It has 471468 ...
user avatar
9 votes
Accepted

Speech audio files dataset with language labels

You can use the Tatoeba website which has full sentences in text and audio as downloads. Sentences with audio Download http://downloads.tatoeba.org/exports/sentences_with_audio.tar.bz2 Fields and ...
user avatar
  • 17.1k
8 votes
Accepted

List of abbreviations and acronyms

I found a github repository that scrapes the wiki list of roughly 6000 acronyms https://github.com/krishnakt031990/Crawl-Wiki-For-Acronyms raw acronyms file and then I wrote this python script to ...
user avatar
  • 17.1k
8 votes

Speech audio files dataset with language labels

I suggest looking at the SpokenLanguages2 dataset, which is very extensive, containing a bit more than one hour of speech for each of the 172 languages: https://community.topcoder.com/longcontest/?...
user avatar
  • 81
8 votes

Database world cities with their names in all languages of the world

I guess the Geonames project is the closest answer to your question. You can download the data directly on the website or you can also find it here and download it/use the api. Cheers
user avatar
  • 1,137
7 votes

List of English words

If you're on a Mac, there's a list of words, 1 per line, in the file /usr/share/dict/words. The usr folder is hidden by default, but you can view it by following the following steps: Open the Finder. ...
user avatar
  • 71
7 votes

Speech audio files dataset with language labels

UCLA Phonetics Lab Archive offers many audio files of spoken word in many languages, although you'll have to dig through and mix-match to your liking: http://archive.phonetics.ucla.edu/ IPA Handbook ...
user avatar
  • 11.8k
7 votes

Speech audio files dataset with language labels

Maybe you can do something with https://rhinospike.com/language/ It's a bit like Tatoeba mentioned above edit : Take also a look at librivox (public domain audiobooks). It comes with an API, then you ...
user avatar
  • 131
7 votes
Accepted

Audio Datasets Featuring Different Speakers Saying the Same Sentence (English)?

The speech accent archive: uniformly presents a large set of speech samples from a variety of language backgrounds. As for English, the following subset contains the sentences: Please call ...
user avatar
6 votes

Dataset of languages and where they are spoken (sub-national)

The CIA Word Factbook has a field listing for languages spoken per country and percentage of population that speaks it. https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html ...
user avatar
6 votes

Database of English words pronunciation

Take a look at Project SHTOOKA
user avatar
6 votes
Accepted

Database of English words difficulty

The New General Service List (updated just a few weeks ago) is a list of "the most important words for second language learners of English". It ranks the top three thousand headwords (plus more words ...
user avatar
  • 737
6 votes

Is a spoken digit dataset available?

https://github.com/Jakobovski/free-spoken-digit-dataset is a free spoken digit dataset (FSDD). As an added bonus it comes with a few useful python utility functions. I created this dataset because I ...
user avatar
6 votes

Speech audio files dataset with language labels

I've found something else. It is called the wide language index. See on github. It consists in a listing of radio podcasts. It's pretty impressive in the numbers of languages covered. Note : It was ...
user avatar
  • 131
6 votes
Accepted

How to collect hand-writting data

The first step is splitting the image into character arrays. To do that, check out the answers in this question: Separate image of text into component character images. In particular, the ImageMagick ...
user avatar
  • 17.1k
6 votes
Accepted

Is there any picture corpus?

You may be able to use the International Picture Naming Project (IPNP), but I think the total pictures are way less than 10k. Here's a link to the query tool. You may get lucky searching through "...
user avatar
  • 17.1k
6 votes

Where can I find a list of latin words?

You can get 8600 Latin words from Latin wikitionary or 879.898 Latin words from English wiktionary using PetScan. You can change output to csv or tsv in the "Output" tab.
user avatar
  • 381
5 votes

List of English words

If you're trying to use Google, the general terms to use would be : From the linguisitics community: 'text corpora' or 'text corpus' (http://en.wikipedia.org/wiki/List_of_text_corpora) or For the ...
user avatar
  • 4,485
5 votes

Phones dataset for speech recognition (not telephone number)

You can find US and British english language phones at university linguistic departments such as Berkeley, Ca. Though, what is useful for speech recognition are the features in statistical acoustic ...
user avatar
  • 131
5 votes

Data set of news articles and scientific journals

As far as I know, one of the best data sources for NLP has been the ENRON emails. While this is not a "news" source, it would certainly provide some raw data for you to play around with. Another ...
user avatar
5 votes

List of abbreviations and acronyms

There doesn't seem to be a proper open acronym project, unfortunately. Data source The best solution at the moment seems to be Wiktionary, which has a category called Initialism (a synonym for ...
user avatar
  • 17.1k
5 votes

Are there opendatasets of English terms describing nationalities, ethnicities?

WordNet is a (free) semantic database of the English language. You could query for example the complete hyponym tree of the term "inhabitant". Depending on your application, it could be a disadvantage ...
user avatar
  • 406
5 votes

Where can I find a part-of-speech corpus free for commercial use?

After doing some searching it seems that the Moby Project is in the public domain, and they have a POS corpus. However, it's simply a dictionary so it doesn't help with words that have multiple POS. ...
user avatar
5 votes

Single-word Translations in All Languages

I'm not aware of an available dataset in the requested form, but it should be possible to obtain something similar by parsing data from wiktionary. The latest dump can be obtained at https://dumps....
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible