https://github.com/dkealvaro/dumbgpt
If you tried watching Karpathy's 2h minGPT video and thought, "I need something even simpler, I dont understand this", this repo is for you.
https://github.com/dkealvaro/dumbgpt
ai gpt llms nlp pytorch
Last synced: 8 months ago
JSON representation
If you tried watching Karpathy's 2h minGPT video and thought, "I need something even simpler, I dont understand this", this repo is for you.
- Host: GitHub
- URL: https://github.com/dkealvaro/dumbgpt
- Owner: DKeAlvaro
- Created: 2025-02-06T22:21:26.000Z (8 months ago)
- Default Branch: master
- Last Pushed: 2025-02-12T17:45:51.000Z (8 months ago)
- Last Synced: 2025-02-21T05:16:32.516Z (8 months ago)
- Topics: ai, gpt, llms, nlp, pytorch
- Language: Python
- Homepage:
- Size: 1.01 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# dumbGPT
If you tried to watch Karpathy's 2h GPT video and thought, "I need something simpler, I dont even understand this. Why is everything so difficult?", this is the repo for you.
## How to use this repo
- `main.py` has a full walkthrough from data processing until model inference Start from there.
- `inputs` directory contains 27 .txt files with 100 observations each,and we assume all those observations come from the same distribution. We want to train our model to be able to learn these sequences
- `src\dumbGPT.py` is the simplest implementation of a GPT model from [Karpathy's mingpt implementation](https://github.com/karpathy/minGPT)
- `tokenizer.py` is a simple float tokenizer for our specific use case