Training

• Benchmark

  • ASQI Engineer - run containerised AI tests and map to score cards!   
  • MLE-bench - bench is a benchmark for measuring how well AI agents perform at machine learning engineering   
  • skill-optimizer - optimize SDK/CLI/MCP guidance so every agent model can use your tool reliably.   

• FineTune

  • LLaMa-Factory - Tuning of 100+ LLMs & VLMs (ACL 2024).   
  • Swift - parameter to finetune 450+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek-R1, ...) and 150+ MLLMs (Qwen2.5-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL2, Phi3.5-Vision, GOT-OCR2, ...).   
  • Axolotl - ai-cloud/axolotl.svg?style=flat&color=green)  
  • torchtune - training library.   
  • unsloth - R1 & Reasoning LLMs 2x faster with 70% less memory! 🦥   
  • maestro - tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL.   
  • EasyLM - training, finetuning, evaluating and serving LLMs in JAX/Flax.   
  • LMFlow - commit/optimalscale/lmflow?color=green)
  • MLX-VLM - VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.   
  • Transformer Lab - tune, and evaluate large language models on your own computer.   

• Framework

  • ColossalAI - commit/hpcaitech/ColossalAI?color=green)
  • Ludwig - code framework for building custom LLMs, neural networks, and other AI models.   
  • MLX - explore/mlx.svg?style=flat&color=green)  
  • Candle - commit/huggingface/candle?color=green)
  • DLRover - machine-learning/dlrover.svg?style=flat&color=green)  
  • AXLearn - commit/apple/axlearn?color=green)
  • MaxText - commit/google/maxtext?color=green)

• RLHF

  • Self-RLHF - alignment/safe-rlhf.svg?style=flat&color=green)  
  • OpenRLHF - to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT).   

• Workflow

  • Flyte - commit/flyteorg/flyte?color=green)
  • Kubeflow - commit/kubeflow/kubeflow?color=green)
  • Metaflow - commit/netflix/metaflow?color=green)
  • MLflow - commit/mlflow/mlflow?color=green)
  • Seldon-Core - core.svg?style=flat&color=green)  
  • ZenML - io/zenml.svg?style=flat&color=green)  
  • Ray - project/ray.svg?style=flat&color=green)  
  • Polyaxon - commit/polyaxon/polyaxon?color=green)