https://github.com/khoaguin/rupygpt

ChatGPT character-level language modeling from scratch in Python and Rust
https://github.com/khoaguin/rupygpt

Last synced: 11 days ago
JSON representation

ChatGPT character-level language modeling from scratch in Python and Rust

• Host: GitHub
• URL: https://github.com/khoaguin/rupygpt
• Owner: khoaguin
• Created: 2024-05-25T10:56:25.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-06-09T06:26:19.000Z (5 months ago)
• Last Synced: 2024-10-06T12:24:17.656Z (about 1 month ago)
• Language: Jupyter Notebook
• Size: 4.81 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# RuPyGPT



    



ChatGPT character-level language modeling from scratch in Python and Rust: 



    



where `18, 47...` are integer encodings of characters, e.g. `18` encodes `F`, `47` encodes `i`.

## Python Requirements

- Python 3.12

- `pip install uv`

- `uv pip install torch numpy transformers datasets tiktoken wandb tqdm`

## References

- [Neural Network: Zero to Hero (Andrej Karpathy)](https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ)

- [NanoGPT](https://github.com/karpathy/nanoGPT)