I have downloaded all the single words from Google Books ngram data (http://storage.googleapis.com/books/ngrams/books/datasetsv2.html) and I could write code to aggregate it all generate word frequencies. However I have seen it mentioned (e.g. https://www.frontiersin.org/articles/10.3389/fpsyg.2011.00027/full) that Google has published word frequency data besides all the raw words. I am unable to find this data. Does anyone know where I can find already computed word frequencies for the Google Books data?
Add a comment
|
1 Answer
As you mentioned, the raw data can easily be parsed to give exactly what you are looking for, and in exactly the format you are looking for. e.g. https://gitlab.com/jberkel/google-ngram-word-frequency-lists
But for pre-made frequency lists, from a random google search, check out https://www.ngrams.info/
Free lists
1 million most frequent 2, 3, 4, and 5-grams
Inexpensive data sets
All n-grams that occur three times or more: 6.2 million 2-grams, 11.9 million 3-grams, and 8.3 million 4-grams
All 2, 3, and 4-grams
Up to 155 million distinct strings -- searchable by word form and part of speech (as above), and also lemma
(download require registration)
