https://github.com/balaji1233/llm-fine-tuning
LLM Fine-Tuning Examples
https://github.com/balaji1233/llm-fine-tuning
fine-tuning llama-factory llm
Last synced: 3 months ago
JSON representation
LLM Fine-Tuning Examples
- Host: GitHub
- URL: https://github.com/balaji1233/llm-fine-tuning
- Owner: balaji1233
- Created: 2025-01-28T11:22:42.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-01-29T18:30:32.000Z (8 months ago)
- Last Synced: 2025-03-19T17:09:40.433Z (7 months ago)
- Topics: fine-tuning, llama-factory, llm
- Language: Jupyter Notebook
- Homepage:
- Size: 494 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM-Fine-Tuning
LLaMA Factory is an easy-to-use and efficient large language model training and fine-tuning platform. With LLaMA Factory, you can fine-tune hundreds of pre-trained models locally without writing any code. The framework features include:
Model types: LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen, Yi, Gemma, Baichuan, ChatGLM, Phi, etc.
Training algorithms: (incremental) pre-training, (multimodal) instruction-supervised fine-tuning, reward model training, PPO training, DPO training, KTO training, ORPO training, etc.
Operation precision: 16-bit full parameter fine-tuning, frozen fine-tuning, LoRA fine-tuning, and 2/3/4/5/6/8-bit QLoRA fine-tuning based on AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ.
Optimized algorithms: GaLore, BAdam, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, and PiSSA.
Acceleration operators: FlashAttention-2 and Unsloth.
Inference engines: Transformers and vLLM.
Experimental panels: LlamaBoard, TensorBoard, Wandb, MLflow, etc.
Github : https://github.com/hiyouga/LLaMA-Factory/tree/main
Doc : https://llamafactory.readthedocs.io/zh-cn/latest/