4

I am looking for a word dataset composed by lists of words of multiple languages (possibly all the major languages), non topic specific, divided by language.

I don't really care for translations, meanings, alternative forms, pronunciation or anything else. However, I would need an offline dataset.

Do you know if any such dataset exist?

ps: I have seen the Multilingual Wordnet site at http://compling.hss.ntu.edu.sg/omw/ however it is missing some important languages like German, Russian and Korean.

4

If you are just interested in words (and not translations) you can check out Wortschatz at Universität Leipzig.

An alternative (with very different coverage across languages, but actually providing translations) is the Wiktionary.

  • 1
    Thanks the list of words from Wortschatz are exactly what I wanted, few extra information and easily parseable – ClonedOne Jul 25 '17 at 18:39
  • 1
    There are also python bindings for their website available at github.com/lehmannro/libleipzig-python – Suzana Dec 19 '18 at 10:17
0

Consider wikidata. Wikidata is a site that linked all different articles from different wikipedia together, as such article name of these wikipedia articles would form a list of word for each language that might be suitable for your list.

Wikidata database can be downloaded from https://www.wikidata.org/wiki/Wikidata:Database_download

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.