https://github.com/argilla-io/notus

Notus is a collection of fine-tuned LLMs using SFT, DPO, SFT+DPO, and/or any other RLHF techniques, while always keeping a data-first approach

alignment-handbook dpo fine-tuning lm-alignment preference-data trl zephyr

Last synced: 17 Jun 2025