https://github.com/p1atdev/auraflow-ft
https://github.com/p1atdev/auraflow-ft
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/p1atdev/auraflow-ft
- Owner: p1atdev
- License: apache-2.0
- Created: 2024-12-21T15:18:44.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-01-17T03:40:29.000Z (4 months ago)
- Last Synced: 2025-01-17T04:29:21.060Z (4 months ago)
- Language: Python
- Size: 688 KB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# AuraFlow finetuning scripts
WIP
## Features
(planned)
- [NF4 model](https://huggingface.co/p1atdev/AuraFlow-v0.3-bnb-nf4) loading
- QLoRA with bitsandbytes
- [Flash Attention](https://github.com/Dao-AILab/flash-attention) support
- Aspect ratio bucketing
## Setup
```bash
uv sync
```
## Train
Distributed training is not tested yet.
Simplest example:
```bash
accelerate launch \
./main.py \
--config ./configs/lora.yaml
```
> [!WARNING]
> This may be changed in the future.
## References
- https://github.com/kohya-ss/sd-scripts
- Heavily inspired by this repository.
- https://github.com/cloneofsimo/minRF
- Model implementation
- https://github.com/microsoft/LoRA
- https://github.com/huggingface/peft
- Peft logic and implementation
- https://github.com/huggingface/diffusers
- https://github.com/Lightning-AI/pytorch-lightning
- Traning cycle and API design
- https://github.com/NovelAI/novelai-aspect-ratio-bucketing
- Aspect ratio bucketing
- https://github.com/bitsandbytes-foundation/bitsandbytes
- Quantization