https://axolotl-ai-cloud.github.io/axolotl/

Go ahead and axolotl questions
https://axolotl-ai-cloud.github.io/axolotl/

Last synced: 2 months ago
JSON representation

Go ahead and axolotl questions

• Host: GitHub
• URL: https://axolotl-ai-cloud.github.io/axolotl/
• Owner: axolotl-ai-cloud
• License: apache-2.0
• Created: 2023-04-14T04:25:47.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2025-02-05T18:01:59.000Z (2 months ago)
• Last Synced: 2025-02-05T18:06:30.662Z (2 months ago)
• Language: Python
• Homepage: https://axolotl-ai-cloud.github.io/axolotl/
• Size: 8.52 MB
• Stars: 8,475
• Watchers: 47
• Forks: 939
• Open Issues: 274
• Metadata Files:
  • Readme: README.md
  • Contributing: .github/CONTRIBUTING.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE
  • Code of conduct: .github/CODE_OF_CONDUCT.md
  • Security: .github/SECURITY.md
  • Support: .github/SUPPORT.md

Awesome Lists containing this project

• awesome-LLM-resourses - Axolotl - tune a model, run model inference or evaluation, and much more. (微调 Fine-Tuning)

README

        


    

        

        

        

    





    

    

    

    


    

    

    


    

    

    


    

    

    

    

  



Axolotl is a tool designed to streamline post-training for various AI models.

Post-training refers to any modifications or additional training performed on

pre-trained models - including full model fine-tuning, parameter-efficient tuning (like

LoRA and QLoRA), supervised fine-tuning (SFT), instruction tuning, and alignment

techniques. With support for multiple model architectures and training configurations,

Axolotl makes it easy to get started with these techniques.

Axolotl is designed to work with YAML config files that contain everything you need to

preprocess a dataset, train or fine-tune a model, run model inference or evaluation,

and much more.

Features:

- Train various Huggingface models such as llama, pythia, falcon, mpt

- Supports fullfinetune, lora, qlora, relora, and gptq

- Customize configurations using a simple yaml file or CLI overwrite

- Load different dataset formats, use custom formats, or bring your own tokenized datasets

- Integrated with [xformers](https://github.com/facebookresearch/xformers), flash attention, [liger kernel](https://github.com/linkedin/Liger-Kernel), rope scaling, and multipacking

- Works with single GPU or multiple GPUs via FSDP or Deepspeed

- Easily run with Docker locally or on the cloud

- Log results and optionally checkpoints to wandb, mlflow or Comet

- And more!

## 🚀 Quick Start

**Requirements**:

- NVIDIA GPU (Ampere or newer for `bf16` and Flash Attention) or AMD GPU

- Python ≥3.10

- PyTorch ≥2.4.1

### Installation

```shell

pip3 install --no-build-isolation axolotl[flash-attn,deepspeed]

# Download example axolotl configs, deepspeed configs

axolotl fetch examples

axolotl fetch deepspeed_configs  # OPTIONAL

```

Other installation approaches are described [here](https://axolotl-ai-cloud.github.io/axolotl/docs/installation.html).

### Your First Fine-tune

```shell

# Fetch axolotl examples

axolotl fetch examples

# Or, specify a custom path

axolotl fetch examples --dest path/to/folder

# Train a model using LoRA

axolotl train examples/llama-3/lora-1b.yml

```

That's it! Check out our [Getting Started Guide](https://axolotl-ai-cloud.github.io/axolotl/docs/getting-started.html) for a more detailed walkthrough.

## ✨ Key Features

- **Multiple Model Support**: Train various models like LLaMA, Mistral, Mixtral, Pythia, and more

- **Training Methods**: Full fine-tuning, LoRA, QLoRA, and more

- **Easy Configuration**: Simple YAML files to control your training setup

- **Performance Optimizations**: Flash Attention, xformers, multi-GPU training

- **Flexible Dataset Handling**: Use various formats and custom datasets

- **Cloud Ready**: Run on cloud platforms or local hardware

## 📚 Documentation

- [Installation Options](https://axolotl-ai-cloud.github.io/axolotl/docs/installation.html) - Detailed setup instructions for different environments

- [Configuration Guide](https://axolotl-ai-cloud.github.io/axolotl/docs/config.html) - Full configuration options and examples

- [Dataset Guide](https://axolotl-ai-cloud.github.io/axolotl/docs/dataset-formats/) - Supported formats and how to use them

- [Multi-GPU Training](https://axolotl-ai-cloud.github.io/axolotl/docs/multi-gpu.html)

- [Multi-Node Training](https://axolotl-ai-cloud.github.io/axolotl/docs/multi-node.html)

- [Multipacking](https://axolotl-ai-cloud.github.io/axolotl/docs/multipack.html)

- [FAQ](https://axolotl-ai-cloud.github.io/axolotl/docs/faq.html) - Frequently asked questions

## 🤝 Getting Help

- Join our [Discord community](https://discord.gg/HhrNrHJPRb) for support

- Check out our [Examples](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/) directory

- Read our [Debugging Guide](https://axolotl-ai-cloud.github.io/axolotl/docs/debugging.html)

- Need dedicated support? Please contact [✉️[email protected]](mailto:[email protected]) for options

## 🌟 Contributing

Contributions are welcome! Please see our [Contributing Guide](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.github/CONTRIBUTING.md) for details.

## Supported Models

|             | fp16/fp32 | lora | qlora | gptq | gptq w/flash attn | flash attn | xformers attn |

|-------------|:----------|:-----|-------|------|-------------------|------------|--------------|

| llama       | ✅         | ✅    | ✅     | ✅             | ✅                 | ✅          | ✅            |

| Mistral     | ✅         | ✅    | ✅     | ✅             | ✅                 | ✅          | ✅            |

| Mixtral-MoE | ✅         | ✅    | ✅     | ❓             | ❓                 | ❓          | ❓            |

| Mixtral8X22 | ✅         | ✅    | ✅     | ❓             | ❓                 | ❓          | ❓            |

| Pythia      | ✅         | ✅    | ✅     | ❌             | ❌                 | ❌          | ❓            |

| cerebras    | ✅         | ✅    | ✅     | ❌             | ❌                 | ❌          | ❓            |

| btlm        | ✅         | ✅    | ✅     | ❌             | ❌                 | ❌          | ❓            |

| mpt         | ✅         | ❌    | ❓     | ❌             | ❌                 | ❌          | ❓            |

| falcon      | ✅         | ✅    | ✅     | ❌             | ❌                 | ❌          | ❓            |

| gpt-j       | ✅         | ✅    | ✅     | ❌             | ❌                 | ❓          | ❓            |

| XGen        | ✅         | ❓    | ✅     | ❓             | ❓                 | ❓          | ✅            |

| phi         | ✅         | ✅    | ✅     | ❓             | ❓                 | ❓          | ❓            |

| RWKV        | ✅         | ❓    | ❓     | ❓             | ❓                 | ❓          | ❓            |

| Qwen        | ✅         | ✅    | ✅     | ❓             | ❓                 | ❓          | ❓            |

| Gemma       | ✅         | ✅    | ✅     | ❓             | ❓                 | ✅          | ❓            |

| Jamba       | ✅         | ✅    | ✅     | ❓             | ❓                 | ✅          | ❓            |

✅: supported

❌: not supported

❓: untested

## ❤️ Sponsors

Thank you to our sponsors who help make Axolotl possible:

- [Modal](https://www.modal.com?utm_source=github&utm_medium=github&utm_campaign=axolotl) - Modal lets you run

jobs in the cloud, by just writing a few lines of Python. Customers use Modal to deploy Gen AI models at large scale,

fine-tune large language models, run protein folding simulations, and much more.

Interested in sponsoring? Contact us at [[email protected]](mailto:[email protected])

## 📜 License

This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.