1

The Pile An 800GB Dataset of Diverse Text for Language Modeling What is the Pile? The Pile is an 825 GiB diverse, open-source language modeling data set that consists of 22 smaller, high-quality datasets combined together. Why is the Pile a good training set? Recent work has shown that especially for large models, diversity in data sources improves general ...


Only top voted, non community-wiki answers of a minimum length are eligible