Known datasets for long document analysis

Question

Are there good baselines datasets or benchmarks for similarities/retrieval of long texts (extremely long documents, books, etc.)? Although significant both academically and applicational, I cannot get a hold of any significant dataset nor remarkable papers from the last AAA conferences.

Daily Mail, CNN or IMDB reviews are too short for me.

Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles(2020) or Graph Convolutional Networks for Text Classification(2018)

For example, state to work on long text, but only use the first paragraph of the article.

My direction would probably be Wikipedia or Arvix datasets, but I would be happy to hear your experience or to find a relevant baseline to compete against.

Answer 1

I'm not an expert on nlp but if your looking for lengthy machine readable corpora you may want to take a look at the HathiTrust project. It's a University lead initiative to make publicly available the text of millions of digitized items.

They have an analytics platform where you can start running code and queries through your web browser but you need a University affiliation to make use of it.