https://github.com/jcolechanged/sage-rl
https://github.com/jcolechanged/sage-rl
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/jcolechanged/sage-rl
- Owner: jColeChanged
- Created: 2024-05-11T20:45:40.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-18T20:21:44.000Z (about 2 years ago)
- Last Synced: 2025-01-25T23:03:31.096Z (over 1 year ago)
- Language: Python
- Size: 96.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Finetuning and RLHF scripts for the [Guild of the Rose](https://guildoftherose.org/)
Sage AI. Based off the HuggingFace [TRL library](https://github.com/huggingface/trl).
## Getting Started
Step 1. Install [CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/)
Step 2. Install [Python](https://www.python.org/)
Step 3. Create a virtual environment and install dependencies into it.
```python
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
```
Step 4. Before you can do training you'll need to configure your accelerate config.
```bash
accelerate config
```
Once the accelerator is in place you can do supervised fine-tuning with:
```bash
trl sft --model_name_or_path facebook/opt-125m --dataset_name imdb --output_dir opt-sft-imdb
```
Direct policy optimization with:
```bash
trl dpo --model_name_or_path facebook/opt-125m --output_dir trl-hh-rlhf --dataset_name trl-internal-testing/hh-rlhf-trl-style
````