{"id":25288973,"url":"https://github.com/horus-ai-labs/DistillFlow","last_synced_at":"2025-10-27T22:30:46.971Z","repository":{"id":258876540,"uuid":"869239592","full_name":"horus-ai-labs/DistillFlow","owner":"horus-ai-labs","description":null,"archived":false,"fork":false,"pushed_at":"2025-02-10T06:12:26.000Z","size":34838,"stargazers_count":34,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-10T06:29:24.431Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/horus-ai-labs.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-08T00:49:14.000Z","updated_at":"2025-02-10T06:12:28.000Z","dependencies_parsed_at":"2024-12-13T17:21:15.106Z","dependency_job_id":"e7214de0-854d-4563-ac65-206006b1a675","html_url":"https://github.com/horus-ai-labs/DistillFlow","commit_stats":null,"previous_names":["horus-ai-labs/distillflow"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horus-ai-labs%2FDistillFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horus-ai-labs%2FDistillFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horus-ai-labs%2FDistillFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horus-ai-labs%2FDistillFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/horus-ai-labs","download_url":"https://codeload.github.com/horus-ai-labs/DistillFlow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238563734,"owners_count":19492975,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-12T23:01:52.500Z","updated_at":"2025-10-27T22:30:46.966Z","avatar_url":"https://github.com/horus-ai-labs.png","language":"Python","funding_links":[],"categories":["others","Python"],"sub_categories":[],"readme":".. image:: https://s3.us-west-2.amazonaws.com/www.horusailabs.com/distillflow.png\n    :target: https://horusailabs.com/\n    :align: center\n========\n\n.. raw:: html\n\n   \u003cdiv align=\"center\"\u003e\n     \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/horus-ai-labs/DistillFlow\"/\u003e\u003c/a\u003e\n     \u003ca href=\"https://github.com/horus-ai-labs/DistillFlow/discussions\"\u003e\u003cimg src=\"https://img.shields.io/badge/status-beta-red.svg\"/\u003e\u003c/a\u003e\n     \u003ca href=\"https://www.python.org/downloads/release/python-3120/\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.12-green.svg\"/\u003e\u003c/a\u003e\n   \u003c/div\u003e\n\nOverview\n========\n\nDistillFlow is an open-source toolkit designed to simplify and scale the distillation of large language models (LLMs) into smaller, more efficient models. It provides a flexible pipeline for distillation, fine-tuning, and experimentation across multiple GPUs, with support for dynamic resource allocation and easy integration of custom techniques.\n\n**What is distillation?**\n\nDistillation is the process of transferring knowledge from large machine learning models to small models. The large model is often called the teacher model while the smaller model is called the student model.\n\n\nArchitecture\n============\nDistillFlow lets you build a fully configurable pipeline, to help with your Distillation.\nOnce the data is available, choose a teacher model, and the student model and your dataset\nand finally run the distillation.\n\n.. raw:: html\n\n   \u003cp align=\"center\"\u003e\n     \u003cimg src=\"https://s3.us-west-2.amazonaws.com/www.horusailabs.com/distillflow_arch.png\" height=\"600\"\u003e\n   \u003c/p\u003e\n\nKey Features\n============\n- **Multi-Strategy Distillation:** Supports multiple distillation techniques such as logits, attention and layers based distillation.\n- **Dynamic Resource Allocation:** Automatically distributes tasks across GPUs or nodes based on available memory.\n- **Fine-Tuning Support:** Allows for domain-specific and downstream fine-tuning of distilled models.\n- **Model Loading Optimizations:** Supports optimized model loading using Unsloth, Liger Kernel, Flash Attention etc.\n- **Easy Integration:** Compatible with popular libraries like Hugging Face Transformers, PyTorch, and DeepSpeed.\n\nRequirements\n============\n\n* Python 3.12+\n* Works on Linux, macOS\n\nInstall\n=======\n\nClone the repository:\n\n.. code-block:: bash\n\n    git clone git@github.com:horus-ai-labs/DistillFlow.git\n    cd DistillFlow\n    pip3 install poetry\n    poetry install\n\nData\n======\nWe support any HuggingFace dataset in ShareGPT or Alpaca formats.\n\nQuick Start\n===========\nHere's a quick example to get started with DistillFlow:\n\nCreate a training config, specifying your teacher model, student model,\nhuggingface dataset and the distillation type.\n\nUse one of the existing test configs in `config` folder. The `local_distill.yaml`\nworks for Mac.\n\nHere is a quick config to get started:\n\n.. code-block:: yaml\n\n    student_model:\n        model_name_or_path: Qwen/Qwen2-1.5B\n    teacher_model:\n        model_name_or_path: Qwen/Qwen2-7B\n    data:\n      text_field: \"text\"\n      train_datasets:\n        - path: mlabonne/FineTome-100k\n          template: sharegpt\n    distill:\n        type: logits\n        max_seq_length: 1024\n        sft_config:\n            output_dir: './results'\n            num_train_epochs: 3\n            per_device_train_batch_size: 1\n            gradient_accumulation_steps: 8\n            eval_strategy: steps\n            eval_steps: 100\n            save_steps: 2000\n            learning_rate: 2.0e-5\n            weight_decay: 0.05\n            warmup_ratio: 0.1\n            lr_scheduler_type: 'cosine'\n            max_grad_norm: 1.0\n            group_by_length: False\n      distillation_args:\n            temperature: 2.0\n            alpha: 0.5\n\nRun the command:\n\n.. code-block:: bash\n\n    accelerate launch src/trainer.py --config \u003cyour_config_path\u003e\n\nAcknowledgement\n=======\nThe repo structure is inspired by `LLamaFactory \u003chttps://github.com/hiyouga/LLaMA-Factory\u003e`_.\nThe distillation training techniques are inspired by the works of `DistillKit \u003chttps://github.com/arcee-ai/DistillKit\u003e`_.\n\n\nLicense\n=======\nDistributed under the Apache-2.0 License. See `LICENSE \u003chttps://github.com/horus-ai-labs/DistillFlow/blob/main/LICENSE\u003e`_ for more information.\n\nCommunity and Support\n=====================\n- Join the discussion on our `GitHub Discussions \u003chttps://github.com/horus-ai-labs/DistillFlow/discussions\u003e`_.\n- Report issues and request features using our `Issue Tracker \u003chttps://github.com/horus-ai-labs/DistillFlow/issues\u003e`_.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorus-ai-labs%2FDistillFlow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhorus-ai-labs%2FDistillFlow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorus-ai-labs%2FDistillFlow/lists"}