https://github.com/nutanixdev/nugpt

Codebase for developing a miniaturized GPT model, following GPT-2 architecture
https://github.com/nutanixdev/nugpt

Last synced: 11 days ago
JSON representation

Codebase for developing a miniaturized GPT model, following GPT-2 architecture

• Host: GitHub
• URL: https://github.com/nutanixdev/nugpt
• Owner: nutanixdev
• License: apache-2.0
• Created: 2023-09-25T04:12:51.000Z (over 2 years ago)
• Default Branch: master
• Last Pushed: 2023-12-21T18:38:04.000Z (over 2 years ago)
• Last Synced: 2026-04-21T23:10:00.658Z (about 1 month ago)
• Language: Python
• Size: 94.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 9
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# nugpt

This repo includes pertinent codebase for developing a miniaturized GPT model. It follows GPT-2 architecture. 

# Commands

## Data Engineering

* Include text data

* run: `python data/data/prepare.py`

## Model Training

* `python train.py --batch_size=32 --wandb_log=True`

## Inference

* `python sample.py --out_dir=out-wiki`