Hot answers tagged

18

ConceptNet is a semantic network containing lots of things computers should know about the world, especially when understanding text written by people. Trying to reproduce your relation-sequences yields: finger PartOf hand IsA body part, which looks surprisingly "dead end". chair IsA seat RelatedTo furniture MadeOf wood ...


14

There are plenty of open corpora (databases) of English words available. Specifically, take a look at Brown and WordNet. Check out Natural Language Toolkit it is written in Python and has those corpora available for download. It is one of the most popular packages to work with human languages data. If you prefer to use web based API, take a look at Wordnik ...


11

English EuroWordNet at globalwordnet.org is worth considering, depending on your requirements. Opencyc.org and Google freebase.com have large taxonomies already available for download.


11

The largest English corpus I've found (over 10,000 messages) is the National University of Singapore's SMS corpus -- select the corpus with "all" messages -- however, closer examination reveals that relatively few of the messages originate from US participants. A corpus of SMS spam messages has been created which are written in English. There are over 1,000 ...


11

The English Language & Usage stackexchange site has a question with answers related to your question What's the largest open-source dictionary that includes brief definitions of each word? Source The popular answer is WordNet from Princeton. You can either browse or download the full data set, although it's about 10 years old. The license allows ...


9

What you're trying to do is called named entity recognition. There exist several typical datasets for it, such as: CoNLL-2002: free. 4 types of entities are tagged: locations, persons, organizations, and miscellaneous entities that do not belong in any of the three previous categories. CoNLL-2003: free. 4 types of entities are tagged: locations, persons, ...


8

Wiktionary is the Wikimedia project's dictionary. Here is an example of a page, although the definitions are still quite technical: https://en.wiktionary.org/wiki/data#English Additionally, like simple.wikipedia, there is a Simple English Wiktionary page. Here is an example: http://simple.wiktionary.org/wiki/data The definition you are looking for: ...


7

Wikimedia Commons currently offers more than 20.000 sound files with English pronunciation, around 1.500 of those with British English pronunciation. All of them are published under an open license. Unfortunately, there are currently no dumps of the media files available. However, there is a page that explains how to reuse the content outside of Wikimedia. ...


6

Try http://en.wiktionary.org – it's free and it has a lot of idioms and phrases.


6

The New General Service List (updated just a few weeks ago) is a list of "the most important words for second language learners of English". It ranks the top three thousand headwords (plus more words within each headword, such as plurals and tenses). So it won't cover all the words in your list, and it's really indicating "usefullness" rather than ...


6

It's not entirely open but the best I've found in this vein is www.wordfrequency.info - I am pretty sure that the 5000 word list is free, with fees for larger lists.


5

WordNet is a (free) semantic database of the English language. You could query for example the complete hyponym tree of the term "inhabitant". Depending on your application, it could be a disadvantage that WordNet also contains historical ethnicities or nationalities. For language names, an excellent database is Glottolog. Once I compiled a dictionary of ...


5

English Age-of-acquisition ratings for 30,000 English words (2012): Springer link Free access List of words Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words (1980): Springer link List of words is within the article, but see also 3. Compilation and comparison of many sources (50,000 words, 2012): Description,...


4

I found that the UCLA Phonetics Lab (Archive) page has word lists with sound samples and IPA spelling. There are no parts of speech unfortunately. e͡ɪ hayed o͡u hoed ɔ/ɑ hawed Links to English Language Index Details They mention that only a fraction of data is online. If this is the right format, I'd contact them and ask to share more.


4

I don't know a corpus, but I know a way to create one. If you know how to program you can use the Facebook API and download all the public facebook status from USA with their comments. Then you can use them as a corpus. Info about Facebook API


4

This particular dictionary's legal notice says that: Users are not entitled to [...] transmit [..] it on any other website without the express permission of Pearson. You must not use data mining, robots, scraping or similar data gathering or extraction methods on any part of this Site without our express prior written consent (my emphasis) So either ...


4

Forvo is a crowdsourced effort to create sound files for every word of every language. The great thing is that a word in a language can have more than 1 sound file. In fact, you will often find sound files created by males and females from various regions with different accents and voices. Pronunciations are ranked by clarity, so you can focus only on ...


4

The cmudict provides phonetic spellings of a sizable number of American English words. The CELEX database is a similar project; you can select which data you want and download wordlists at WebCelex. Part-of-speech data (word class) is also available in CELEX. The CELEX download interface is somewhat frustrating, but you should only need to use it right ...


4

Here is a link to several corpora of commonly misspelled words in several varieties of English - list includes the Birbeck Spelling Error corpus


4

This python package can pluralize words for you, you can just run your entire dictionary through it to get the plural forms: http://www.clips.ua.ac.be/pages/pattern-en#pluralization For this you first need Python, I myself use winpython, but any python installation would do. You can then download the pattern package here: http://www.clips.ua.ac.be/pattern


4

Eurovoc is a large semantic "Multilingual Thesaurus of the European Union"


3

Anki is a flashcard tool for computers and phones. A cool thing is that you can use the decks for other purposes. As on example, the deck "500 Word English Pronunciation with IPA and Audio" (link) contains 500 words formatted like this: Lemma business IPA [ˈbɪznɨs] Audio [non-mp3 audio] Explanation -[ɨ] is either [ɪ] or [ə] Tags To get ...


3

The CMU Pronunciation Dictionary sounds like it would be useful for something like this: http://www.speech.cs.cmu.edu/cgi-bin/cmudict It has pronunciation guides for 134,000 English words and uses the ARPAbet, which is directly mappable to IPA (https://en.wikipedia.org/wiki/Arpabet)


3

The official Google Translate API does not offer dictionary data. (If you must know: Google has traditionally paid for licences for dictionary data from various sources, with varying quality, for the various languages.) So I assume you are scraping and thus on poor legal footing, but I doubt it will be a problem in practice. One great and ever-improving ...


3

Maybe the Macmillan Dictionary API can help you


3

Take a look at Project SHTOOKA


3

I am not 100% sure if the following links are useful for you. Please let me know if you are looking for something else. http://www.iitg.ac.in/ece/emstlab/SRdatabase/introduction.php http://accent.gmu.edu/howto.php http://www.signalprocessingsociety.org/technical-committees/list/sl-tc/spl-nl/2012-05/the-rss2015-speech-corpus/


3

I recently made a corpus available of text messages from Spanish-dominant bilingual young adults in New York City (Bilingual Youth Texts Corpus). It is at www.byts.commons.gc.cuny.edu


3

EF English Proficiency Index (2013) "The EF English Proficiency Index calculates a country’s average adult English skill level using data from two different EF English tests completed by hundreds of thousands of adults every year" ... "The EF EPI third edition was calculated using 2012 test data from about 750,000 test takers. Only countries with a ...


Only top voted, non community-wiki answers of a minimum length are eligible