Skip to main content
3 votes

Open Source huge dataset for Language Modelling

The Pile An 800GB Dataset of Diverse Text for Language Modeling What is the Pile? The Pile is an 825 GiB diverse, open-source language modeling data set that consists of 22 smaller, high-quality ...
Orophile's user avatar
  • 1,751
2 votes

Where can I find a large unprocessed dataset?

Japanese Trade Stats This seems to satisfy your request. There are other data sets on that site if your find this particular one doesn’t.
James Risner's user avatar
1 vote

Open Source huge dataset for Language Modelling

https://commoncrawl.org/. Larger (petabytes) but more difficult to use than The Pile. More on the size: https://commoncrawl.github.io/cc-crawl-statistics/plots/crawlsize OpenWebText falcon-refinedweb
Franck Dernoncourt's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible