https://github.com/rahulunair/tiny_llm_finetuner
LLM finetuning on Intel XPUs - LoRA on intel discrete GPUs
https://github.com/rahulunair/tiny_llm_finetuner
gpu intel intel-arc xpu
Last synced: 11 days ago
JSON representation
LLM finetuning on Intel XPUs - LoRA on intel discrete GPUs
- Host: GitHub
- URL: https://github.com/rahulunair/tiny_llm_finetuner
- Owner: rahulunair
- License: mit
- Created: 2023-05-25T18:25:15.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-10-31T03:57:01.000Z (over 1 year ago)
- Last Synced: 2023-10-31T04:30:35.179Z (over 1 year ago)
- Topics: gpu, intel, intel-arc, xpu
- Language: Python
- Homepage:
- Size: 22.5 KB
- Stars: 10
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
- License: LICENSE
Awesome Lists containing this project
README
### Tiny llm Finetuner for Intel dGPUs
#### Finetuning openLLAMA on Intel discrete GPUS
A finetuner[1](#f1) [2](#f2) for LLMs on Intel XPU devices, with which you could finetune the openLLaMA-3b model to sound like your favorite book.
#### Setup and activate conda env
```bash
conda env create -f env.yml
conda activate pyt_llm_xpu
```
**Warning**: OncePyTorch and intel extension for PyTorch is already setup, then install peft without dependencies as peft requires PyTorch 2.0(not supported yet on Intel XPU devices.)
#### Generate data
Fetch a book from guttenberg (default: pride and prejudice) and generate the dataset.
```python
python fetch_data.py
```
#### Finetune
```bash
python finetune.py --input_data ./book_data.json --batch_size=64 --micro_batch_size=16 --num_steps=300
```
#### Inference
For inference, you can either provide a input prompt, or the model will take a default prompt
##### Without user provided prompt
```bash
python inference.py --infer
```
##### Using your own prompt for inference
```bash
python inference.py --infer --prompt "my prompt"
```
##### Benchmark Inference
```bash
python inference.py --bench
```
1: adapted from: [source](https://github.com/modal-labs/doppel-bot/blob/main/src/finetune.py) [↩](#a1)
2: adapted from: [source](https://github.com/tloen/alpaca-lora/blob/65fb8225c09af81feb5edb1abb12560f02930703/finetune.py) [↩](#a2)