For my side project, I need to get a categorization of as many English words as possible. With "categorization of words" I mean the assignment of categories / topics to each word, as it is done in most language dictionaries, e.g.:
Animals
=======
- lion
- horse
- tiger
- cat
- dog
- mouse
...
- mare
- stallion
- calf
...
Plants
- tree
- plant
- root
- branch
...
Up to now, I am using a language dictionary to get such a categorization, but this has its limits. It works fine for frequent words like "lion" and "cat", but rarer words like "stallion" and "mare" are not included in such topic word lists.
What I need is a thorough, as-complete-as-possible list of English words together with their "category". If a word denotes the name of an animal, it belongs to the category "animals", no matter how rare it is. If a word denotes the name of a mechanical tool, or a mechanical action, it should belong to "Tools" or "Crafts" or whatever.
I tried Wordnet with their "hypernym" feature, but the results are limited too. For example, all adjectives seem to not have hypernyms, making it impossible to classify them.
Some comments:
- Note that a word can belong to more than one category.
- I am looking for open data sources, but I'm also grateful for answers suggesting non-open data.
- Bonus points if the data source has the same for other languages.
(Edit) Wordnet limitations
As per Nicolas' request, here a couple of problems with Wordnet. This is just to indicate why Wordnet is not a real solution to our problem.
- For example, the adjective "salty" (which could have hypernyms auch as "taste", "chemical characteristic"), has the synsets
piquant.s.02,salty.a.02,salty.s.03. However, none of them has any hypernyms. - Other adjectives and adverbs without hypernyms: "useless", "lateral", "friendly", "silly", "creepy", "merely", "creamy", "hot", "yellow", "dreamy", "done". Some of them have hypernyms as nouns, but not as their main meaning as adjectives or adverbs.
- As another example, "Mexico" (
mexico.n.01) has no hypernym that indicates that it's a country's name. Same for (all?) other countries. - "above" has "r", "s" and "n" part-of-speeches. None of them has a hypernym, except the "n" (noun), which has
section.n.01,writing.n.02,written_communication.n.01, i.e. the meaning of "the above" in the sense of "what is written above". This is a very narrow meaning of the word "above". The spatial meaning, which is much more general, is not captured by any hypernym.
I understand that it's not easy to find a coherent ontology for adjectives. However, a basic categorization should be possible:
- salty, sweet, sour, bitter, ... ==> taste
- red, orange, yellow, ... ==> color
- friendly, dreamy, creepy, ... ==> moods
- friendly, silly, ... ==> personal characteristics
- creamy, hot, ... ==> food characteristics (of which "taste" would be a sub category)
- above, bottom, left, ... ==> spatial
