1
The Pile
An 800GB Dataset of Diverse Text for Language Modeling
What is the Pile?
The Pile is an 825 GiB diverse, open-source language modeling data set that consists of 22 smaller, high-quality datasets combined together.
Why is the Pile a good training set?
Recent work has shown that especially for large models, diversity in data sources improves general ...
Only top voted, non community-wiki answers of a minimum length are eligible
Related Tags
data-request × 2900geospatial × 386
usa × 267
medical × 204
machine-learning × 151
api × 123
nlp × 94
finance × 94
data.gov × 91
images × 88
government × 84
weather × 80
language × 79
research × 78
transportation × 77
sports × 74
economics × 73
historical × 68
uk × 67
europe × 65
uses-of-open-data × 64
time-series × 56
business × 53
education × 51
demographics × 49