https://github.com/eleutherai/pile-explorer

For exploring the data and documenting its limitations
https://github.com/eleutherai/pile-explorer

Last synced: 11 months ago
JSON representation

For exploring the data and documenting its limitations

• Host: GitHub
• URL: https://github.com/eleutherai/pile-explorer
• Owner: EleutherAI
• License: mit
• Created: 2020-10-28T16:49:33.000Z (over 5 years ago)
• Default Branch: main
• Last Pushed: 2021-01-21T16:43:55.000Z (over 5 years ago)
• Last Synced: 2025-04-24T18:48:22.381Z (about 1 year ago)
• Language: Python
• Size: 39.1 KB
• Stars: 5
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Codeowners: CODEOWNERS

Awesome Lists containing this project

README

          
# Exploring the Pile

This repository contains code for exploring the Pile and documenting its limitations

## Language Modeling Data Format

The data in the Pile is stored in the [lm_dataformat](https://github.com/leogao2/lm_dataformat). This repository is designed to be used on data stored in that format. For documentation, see the linked repository.