https://github.com/jmaczan/gpt
Generative Pre-trained Transformer in PyTorch from scratch
https://github.com/jmaczan/gpt
attention deep-learning from-scratch gpt machine-learning pytorch transformer
Last synced: 12 months ago
JSON representation
Generative Pre-trained Transformer in PyTorch from scratch
- Host: GitHub
- URL: https://github.com/jmaczan/gpt
- Owner: jmaczan
- License: gpl-3.0
- Created: 2024-06-05T14:29:59.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-01T19:17:02.000Z (over 1 year ago)
- Last Synced: 2025-04-12T13:07:29.583Z (about 1 year ago)
- Topics: attention, deep-learning, from-scratch, gpt, machine-learning, pytorch, transformer
- Language: Python
- Homepage: https://pagedout.institute/download/PagedOut_005.pdf#page=6
- Size: 95.7 KB
- Stars: 3
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
# gpt
> 💜 See also the article how to build the minimalistic GPT version in Paged Out! #5 Issue, page 6 ["GPT in PyTorch"](https://pagedout.institute/download/PagedOut_005.pdf#page=6)
Generative Pre-trained Transformer in PyTorch from scratch
## Train
### CLI
```sh
python src/train.py
```
Options:
```sh
--batch_size 64
--num-epochs 100
--lr 0.0001
--from-checkpoint checkpoint_path.pth
```
Model is checkpointed after each epoch and stored in `checkpoints/` directory
### Code
```py
from train import train
train()
```
## Run
### CLI
```sh
python src/run.py --from-checkpoint checkpoint_path.pth
```
### Code
```py
from run import run
run(model_path="checkpoint_path.pth", prompt="Rick:\nMorty, where are you?)
```
## Cite
If you use this software in your research, please use the following citation:
```bibtex
@misc{Maczan_GPT_2024,
title = "Generative Pre-trained Transformer in PyTorch",
author = "{Maczan, Jędrzej Paweł}",
howpublished = "\url{https://github.com/jmaczan/gpt}",
year = 2024,
publisher = {GitHub}
}
```
## License
GPL v3
## Author
Jędrzej Maczan, 2024