Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/s-macke/gopt
GPT-2 Model Inference
https://github.com/s-macke/gopt
ai artificial-intelligence deep-learning golang gpt-2 transformers
Last synced: 30 days ago
JSON representation
GPT-2 Model Inference
- Host: GitHub
- URL: https://github.com/s-macke/gopt
- Owner: s-macke
- License: mit
- Created: 2023-03-01T22:05:25.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-01-14T11:37:11.000Z (10 months ago)
- Last Synced: 2024-04-16T00:55:08.400Z (7 months ago)
- Topics: ai, artificial-intelligence, deep-learning, golang, gpt-2, transformers
- Language: Go
- Homepage:
- Size: 210 KB
- Stars: 3
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# GoPT
This is my learning project to understand transformer models.
In particular, it implements GPT-2 model inference. It is the same model type
used for ChatGPT, only smaller and therefore less powerful.
The model is implemented in pure Go, without any external dependencies and not optimized for speed.
## Build
```bash
go build
```
## Download and convert small model
Download the model files
* model.safetensors
* vocab.json
from huggingface at
https://huggingface.co/openai-community/gpt2/tree/main
```bash
wget -O model.safetensors "https://huggingface.co/openai-community/gpt2/resolve/main/model.safetensors?download=true"
wget -O vocab.json "https://huggingface.co/openai-community/gpt2/raw/main/vocab.json?download=true"
./GoPT
```
## Credits
This code is based on the following projects:
* https://github.com/viznut/vzgpt (the layer inference part)
* https://bellard.org/libnc/libnc.html (The transformation of python matrices to binary files. The original gpt2tc seems to be no longer available)
* https://github.com/ggerganov/ggml (just for reference)