6

Do you know if there is any (open) database for Google Books titles or texts? It could be used for text analysis and statistics.

4

Google Books publishes an API. https://developers.google.com/books/docs/v1/getting_started

Of course one of the prohibitions in their ToS (https://developers.google.com/terms/) is "you will not...Scrape, build databases or otherwise create permanent copies of such content" so if anyone built an open database from this data, which I think is what you're asking for, it may be in violation of the ToS.

2

Amazon Web Services provide several open dataset for their clients including mathematics, economics, biology, astronomy etc. One of them is Google Books Ngrams. It's not exactly titles dataset but it is a 2.2 TB with Ngrams.

N-grams are fixed size tuples of items. In this case the items are words extracted from the Google Books corpus. The n specifies the number of elements in the tuple, so a 5-gram contains five words or characters.

The yellow dog played fetch.

Would produce the following 2-grams:

["The", "yellow"] ["yellow", 'dog"] ["dog", "played"] ["played", "fetch"] ["fetch", "."]

For more details check this link

1

Project Gutenberg hosts public domain books and texts and offers a bulk download.

There is also a python package to download and clean up texts.


You could also do a bulk download of WikiBooks - see these links for more details

https://dumps.wikimedia.org/

https://dumps.wikimedia.org/enwikibooks/20160203/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.