18

ConceptNet is a semantic network containing lots of things computers should know about the world, especially when understanding text written by people. Trying to reproduce your relation-sequences yields: finger PartOf hand IsA body part, which looks surprisingly "dead end". chair IsA seat RelatedTo furniture MadeOf wood ...


16

Have you seen WikiQuote by the Wikimedia Foundation? There is an API endpoint at http://en.wikiquote.org/w/api.php which uses the standard MediaWiki API for there are API clients in many different languages. EDIT: Two WikiQuote API-specific links: https://stackoverflow.com/questions/13762688/wiki-quotes-api and http://bwgz57.wordpress.com/2013/02/14/in-...


12

You can download an Aspell Dictionary, then convert it to simple list of words: aspell -d en dump master | aspell -l en expand > my.dict A few other dictionaries.


11

English EuroWordNet at globalwordnet.org is worth considering, depending on your requirements. Opencyc.org and Google freebase.com have large taxonomies already available for download.


10

Wikidata has a category called "city" which will give you a list of world cities https://www.wikidata.org/wiki/Q515 You can use SPARQL to query Wikidata https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual https://query.wikidata.org/ A sample SPARQL query will give you the city name in the target language SELECT ?city ?cityLabel ?country ?...


9

Use wordnet, it's an academic project but a de facto for various purposes including the one you asked for. There are also higher semantics modeled that you can use for various functions. It's also convenient to query and integrate it into your app.


9

The cmudict provides phonetic spellings of a sizable number of American English words. The CELEX database is a similar project; you can select which data you want and download wordlists at WebCelex. Part-of-speech data (word class) is also available in CELEX. The CELEX download interface is somewhat frustrating, but you should only need to use it right ...


9

Tatoeba.org is exactly that: http://tatoeba.org Their data is human-edited and used by dozens of products/websites including electronic dictionaries, so it is of reasonable quality. It has 471468 English sentences and 179 languages. (not all sentences are translated to all languages though, far from that) The structure is not a simple 1-1-1, a sentence ...


9

You can use the Tatoeba website which has full sentences in text and audio as downloads. Sentences with audio Download http://downloads.tatoeba.org/exports/sentences_with_audio.tar.bz2 Fields and structure Sentence id File description Contains the ids of the sentences, in all languages, for which audio is available. Thanks to Nicolas Raoul in this ...


8

I found a github repository that scrapes the wiki list of roughly 6000 acronyms https://github.com/krishnakt031990/Crawl-Wiki-For-Acronyms raw acronyms file and then I wrote this python script to read that code's output and clean it up, as well as put it into a valid CSV (gist) #!/usr/bin/env python # coding=utf-8 ignore_list = ('Search-Navigation','...


8

I suggest looking at the SpokenLanguages2 dataset, which is very extensive, containing a bit more than one hour of speech for each of the 172 languages: https://community.topcoder.com/longcontest/?module=ViewProblemStatement&rd=16555&pm=13978 The audio is very high quality. Although they do not provide access to the testing set, you have to do your ...


8

I guess the Geonames project is the closest answer to your question. You can download the data directly on the website or you can also find it here and download it/use the api. Cheers


7

If you're on a Mac, there's a list of words, 1 per line, in the file /usr/share/dict/words. The usr folder is hidden by default, but you can view it by following the following steps: Open the Finder. In the menu, go to Go->Go to Folder or press Cmd-Shift-G. In the box, type /usr/share/dict. You'll see a list of files. Copy the file called words to whatever ...


7

UCLA Phonetics Lab Archive offers many audio files of spoken word in many languages, although you'll have to dig through and mix-match to your liking: http://archive.phonetics.ucla.edu/ IPA Handbook downloads actually seems to be exactly what you are asking for, however its only for personal use; any other usage requires contacting them for permission...so ...


7

Maybe you can do something with https://rhinospike.com/language/ It's a bit like Tatoeba mentioned above edit : Take also a look at librivox (public domain audiobooks). It comes with an API, then you could send a request like https://librivox.org/api/feed/audiobooks?fields={url_zip_file,language}&offset=XXX&limit=YYY to extract files' id


7

The speech accent archive: uniformly presents a large set of speech samples from a variety of language backgrounds. As for English, the following subset contains the sentences: Please call Stella. Ask her to bring these things with her from the store: Six spoons of fresh snow peas, five thick slabs of blue cheese, and maybe a snack for her ...


6

The New General Service List (updated just a few weeks ago) is a list of "the most important words for second language learners of English". It ranks the top three thousand headwords (plus more words within each headword, such as plurals and tenses). So it won't cover all the words in your list, and it's really indicating "usefullness" rather than ...


6

Take a look at Project SHTOOKA


6

The CIA Word Factbook has a field listing for languages spoken per country and percentage of population that speaks it. https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html For finer grain, you can Google 'language spoken home dataset'. This will find census datasets related to the distribution of languages spoken. Below are some ...


6

https://github.com/Jakobovski/free-spoken-digit-dataset is a free spoken digit dataset (FSDD). As an added bonus it comes with a few useful python utility functions. I created this dataset because I had the same problem. Please contribute to increase the dataset's size.


6

I've found something else. It is called the wide language index. See on github. It consists in a listing of radio podcasts. It's pretty impressive in the numbers of languages covered. Note : It was made for the game The Great language game which is fun ;).


6

The first step is splitting the image into character arrays. To do that, check out the answers in this question: Separate image of text into component character images. In particular, the ImageMagick answer from 2015. (If you can determine how the input is given, then collect the characters as separate images.) To convert the image into a 2D array, you can ...


6

You may be able to use the International Picture Naming Project (IPNP), but I think the total pictures are way less than 10k. Here's a link to the query tool. You may get lucky searching through "image recognition" research websites. Here's one compilation of online resources, and a particular one.


6

You can get 8600 Latin words from Latin wikitionary or 879.898 Latin words from English wiktionary using PetScan. You can change output to csv or tsv in the "Output" tab.


5

If you're trying to use Google, the general terms to use would be : From the linguisitics community: 'text corpora' or 'text corpus' (http://en.wikipedia.org/wiki/List_of_text_corpora) or For the password cracking community : 'wordlist' (one word, might need to keep quoted; http://www.openwall.com/wordlists/)


5

Second try: Google Ngram Viewer contains raw counts of 1-, 2-, ...-grams of text, retrieved from its book scanning endeavor. The section 1-grams contains counts of the occurence of lettres, numbers and even punctuation. They are provided as tab-separated value files, so the frequencies should be derivable with modest scripting efforts. Found via Wikipedia ...


5

You can find US and British english language phones at university linguistic departments such as Berkeley, Ca. Though, what is useful for speech recognition are the features in statistical acoustic models that are extracted from a large amount of annotated audio recordings. These acoustic model features involve hidden markov models and Mel-Frequency Cepstral ...


5

As far as I know, one of the best data sources for NLP has been the ENRON emails. While this is not a "news" source, it would certainly provide some raw data for you to play around with. Another source is PubMed. While this doesn't provide you with the actual article information, it does provide you with a lot of abstracts for scientific writing. There have ...


5

There doesn't seem to be a proper open acronym project, unfortunately. Data source The best solution at the moment seems to be Wiktionary, which has a category called Initialism (a synonym for acronym) See, for example https://en.wiktionary.org/wiki/AAAA The good part about Wiktionary.org: multiple languages links to Wikipedia pages (not all, though), ...


Only top voted, non community-wiki answers of a minimum length are eligible