https://github.com/yyyasin19/gpt

https://github.com/yyyasin19/gpt

Last synced: 24 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/yyyasin19/gpt
• Owner: YYYasin19
• Created: 2023-12-01T21:14:32.000Z (11 months ago)
• Default Branch: master
• Last Pushed: 2023-12-23T22:36:45.000Z (11 months ago)
• Last Synced: 2024-04-13T04:21:22.250Z (7 months ago)
• Language: Python
• Size: 453 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Generative Pre-trained Transformer

This repository implements the Decoder part of the [Attention is all you need](https://arxiv.org/pdf/1706.03762.pdf) in pure PyTorch.

The encoding is done using OpenAI's [tiktoken](https://github.com/openai/tiktoken) library.

## Installation

You'll need [pixi](https://github.com/prefix-dev/pixi) or, when you have all the dependencies installed, you can just use pip.

```bash

pixi install # super easy

pip install . --no-deps

```

## Usage

```bash

gpt train --iterations 5000 --text data/tiny-shakespeare.txt

```

this will create a `model.pt` and `config.txt` file.

You can use those to then generate text using

```bash

gpt prompt --model model.pt --config config.txt

```

which simply generates a fixed amount of tokens given the prompt.