https://github.com/lin826/nanogpt-demo
Training and finetuning local GPTs.
https://github.com/lin826/nanogpt-demo
gpt nanogpt numpy pytorch tqdm transformers
Last synced: 3 months ago
JSON representation
Training and finetuning local GPTs.
- Host: GitHub
- URL: https://github.com/lin826/nanogpt-demo
- Owner: lin826
- License: mit
- Created: 2025-11-03T00:08:00.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-11-03T00:28:36.000Z (9 months ago)
- Last Synced: 2025-11-03T02:25:57.819Z (9 months ago)
- Topics: gpt, nanogpt, numpy, pytorch, tqdm, transformers
- Language: Python
- Homepage: https://github.com/karpathy/nanoGPT
- Size: 3.91 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# nanoGPT Reimplement
Reimplement nanoGPT, a local trainable and finetunable agent, into object-oriented style with typing checks.
Instead of using the example `karpathy/chr_rnn/tinyshakespeare/input.txt`, this repo support multiple .txt files from the folder `input/` to be trained with.
## Transformer - model architecture
> Deviation: Unlike the `Add & Norm` after each layer of the original paper, this repo implements pre-norm formulation.
## Getting Started
Create virtual environment and install pyhton libraries.
> Hint: On [the official webpage](https://www.anaconda.com/download), scroll down to find Miniconda Installers section!
### Conda for System Dependencies
```sh
% conda env create -f environment.yml
```
### UV for no-sys-dependent packages
```sh
% uv venv
% uv add -r requirements.txt
```
### [Optional] Compile as a module
```sh
% pip install -e .
```
## Train and Validate
```sh
% pyhon src/main.py
```
### After optional local compile
```sh
% nanogpt
```
## Sample
## Finetune
## Testing
```sh
pytest .
```