{"id":28620946,"url":"https://github.com/KohakuBlueleaf/LyCORIS","last_synced_at":"2025-06-12T05:04:59.298Z","repository":{"id":90447451,"uuid":"607121301","full_name":"KohakuBlueleaf/LyCORIS","owner":"KohakuBlueleaf","description":"Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.","archived":false,"fork":false,"pushed_at":"2025-05-02T13:14:08.000Z","size":222819,"stargazers_count":2353,"open_issues_count":7,"forks_count":158,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-05-28T10:18:57.787Z","etag":null,"topics":["finetune","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KohakuBlueleaf.png","metadata":{"files":{"readme":"README.md","changelog":"Change.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-27T11:05:13.000Z","updated_at":"2025-05-28T03:05:00.000Z","dependencies_parsed_at":"2023-09-22T22:36:01.187Z","dependency_job_id":"d6b97217-ef75-407f-a4ae-f068e7ebb1d9","html_url":"https://github.com/KohakuBlueleaf/LyCORIS","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/KohakuBlueleaf/LyCORIS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohakuBlueleaf%2FLyCORIS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohakuBlueleaf%2FLyCORIS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohakuBlueleaf%2FLyCORIS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohakuBlueleaf%2FLyCORIS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KohakuBlueleaf","download_url":"https://codeload.github.com/KohakuBlueleaf/LyCORIS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KohakuBlueleaf%2FLyCORIS/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259402009,"owners_count":22851866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["finetune","stable-diffusion"],"created_at":"2025-06-12T05:01:24.389Z","updated_at":"2025-06-12T05:04:59.291Z","avatar_url":"https://github.com/KohakuBlueleaf.png","language":"Python","funding_links":[],"categories":["👑Stable Diffusion","\u003cspan id=\"head1\"\u003e *Keywords* \u003c/span\u003e","Image Generation \u0026 Editing"],"sub_categories":["Python","[Reparameterization PEFT](#content)"],"readme":"![pypi](https://img.shields.io/pypi/v/lycoris-lora.svg)\n![versions](https://img.shields.io/pypi/pyversions/lycoris-lora.svg)\n![PyPI - License](https://img.shields.io/pypi/l/lycoris-lora)\n![downloads](https://img.shields.io/pypi/dm/lycoris-lora)\n![commits](https://img.shields.io/github/commit-activity/m/KohakuBlueleaf/LyCORIS/dev)\n![Discord](https://img.shields.io/discord/1082218577395986452)\n\n# LyCORIS - Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion. (ICLR'24)\n\n![banner image](docs/images/banner2.png)\n\nA project that implements different parameter-efficient fine-tuning algorithms for Stable Diffusion.\n\nThis project originated from LoCon (see archive branch).\n\n**If you are interested in discussing more details, you can join [our Discord server](https://discord.gg/VtTFKrj9gJ)**\n\n[![Discord!](https://i.imgur.com/A8tOvFS.jpg)](https://discord.gg/VtTFKrj9gJ)\n\n**If you want to check more in-depth experiment results and discussions for LyCORIS, you can check our [paper](https://openreview.net/forum?id=wfzXa8e783)**\n\n## Algorithm Overview\n\nLyCORIS currently contains LoRA (LoCon), LoHa, LoKr, (IA)^3, DyLoRA, Native fine-tuning (aka dreambooth).\nGLoRA and GLoKr are coming soon.\nPlease check [List of Implemented Algorithms](docs/Algo-List.md) and [Guidelines](docs/Guidelines.md) for more details.\n\nA simple comparison of some of these methods are provided below (to be taken with a grain of salt)\n\n|                       | Full | LoRA | LoHa | LoKr low factor | LoKr high factor $^+$ |\n| --------------------- | ---- | ---- | ---- | --------------- | ---------------------- |\n| Fidelity              | ★   | ●   | ▲   | ◉              | ▲                     |\n| Flexibility $^*$     | ★   | ●   | ◉   | ▲              | ● $^†$              |\n| Diversity             | ▲   | ◉   | ★   | ●              | ★                     |\n| Size                  | ▲   | ●   | ●   | ●              | ★                     |\n| Training Speed Linear | ★   | ●   | ●   | ★              | ★                     |\n| Training Speed Conv   | ●   | ★   | ▲   | ●              | ●                     |\n\n★ \u003e ◉ \u003e ● \u003e ▲\n[\u003e means better and smaller size is better]\n\n$^+$ Usually we take `factor \u003c= 0.5 * sqrt(dim)` as low factor and `factor \u003e= sqrt(dim` as high factor. For example, factor\u003c=8 for SD1.x/SD2.x/SDXL can be seen as low factor, and, factor\u003e=16 can be seen as high factor. \u003cbr\u003e\n$^*$ Flexibility means anything related to generating images not similar to those in the training set, and combination of multiple concepts, whether they are trained together or not \u003cbr\u003e\n$^†$ It may become more difficult to switch base model or combine multiple concepts in this situation \u003cbr\u003e\n\n**The actual performance may vary depending on the datasets, tasks, and hyperparameters used. It is recommended to experiment with different settings to achieve optimal results.**\n\n## Usage\n\n### Image Generation\n\n#### [a1111/sd-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)\n\nAfter sd-webui 1.5.0, LyCORIS models are officially supported by the built-in LoRA system. You can put them in either `models/Lora` or `models/LyCORIS` and use the default syntax `\u003clora:filename:multiplier\u003e` to trigger it.\n\nWhen we add new model types, we will always make sure they can be used with the newest version of sd-webui.\n\nAs for sd-webui with version \u003c 1.5.0 or sd-webui-forge, please check this [extension](https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris).\n\n#### Others\n\nAs far as we are aware, LyCORIS models are also supported in the following interfaces / online generation services (please help us complete the list!)\n\n- [ComfyUI](https://github.com/comfyanonymous/ComfyUI)\n- [InvokeAI](https://github.com/invoke-ai/InvokeAI)\n- [CivitAI](https://civitai.com/)\n- [Tensor.Art](https://tensor.art/)\n\nHowever, newer model types may not always be supported. If you encounter this issue, consider requesting the developers of the corresponding interface or website to include support for the new type.\n\n### Training\n\nThere are three different ways to train LyCORIS models.\n\n- With [kohya-ss/sd-scripts](https://github.com/kohya-ss/sd-scripts) (see a list of compatible graphical interfaces and colabs at the end of the section)\n- With [Naifu-Diffusion](https://github.com/Mikubill/naifu)\n- With your own script by using LyCORIS as standalone wrappers for **ANY** pytorch modules.\n\nIn any case, please install this package in the corresponding virtual environment. You can either install it\n\n- through pip\n\n  ```bash\n  pip install lycoris-lora\n  ```\n- or from source\n\n  ```bash\n  git clone https://github.com/KohakuBlueleaf/LyCORIS\n  cd LyCORIS\n  pip install .\n  ```\n\nA detailed description of the network arguments is provided in [docs/Network-Args.md](docs/Network-Args.md).\n\n#### kohya script\n\nYou can use this package's kohya module to run kohya's training script to train lycoris module for SD models\n\n- with command line arguments\n\n  ```bash\n  accelerate launch train_network.py \\\n    --network_module lycoris.kohya \\\n    --network_dim \"DIM_FOR_LINEAR\" --network_alpha \"ALPHA_FOR_LINEAR\"\\\n    --network_args \"conv_dim=DIM_FOR_CONV\" \"conv_alpha=ALPHA_FOR_CONV\" \\\n    \"dropout=DROPOUT_RATE\" \"algo=locon\" \\\n  ```\n- with `toml` files\n\n  ```bash\n  accelerate launch train_network.py \\\n    --config_file example_configs/training_configs/kohya/loha_config.toml \\\n    --dataset_config example_configs/training_configs/kohya/dataset_config.toml\n  ```\n\n  For your convenience, some example `toml` files for kohya LyCORIS training are provided in [example/training_configs/kohya](example_configs/training_configs/kohya).\n\n#### HCP-Diffusion\n\n**The support for HCP-Diffusion has been dropped on LyCORIS3.0.0, we will wait until HCP side finish the implementation of new wrapper**\n\nYou can use this package's hcp module to run HCP-Diffusion's training script to train lycoris module for SD models\n\n```bash\naccelerate launch -m hcpdiff.train_ac_single \\\n  --cfg example_configs/training_configs/hcp/hcp_diag_oft.yaml\n```\n\nFor your convenience, some example `yaml` files for HCP LyCORIS training are provided in [example/training_configs/hcp](example_configs/training_configs/hcp).\n\nFor the moment being the outputs of HCP-Diffusion are not directly compatible with a1111/sdwebui.\nYou can perform conversion with [tools/batch_hcp_convert.py](tools/batch_hcp_convert.py).\n\nIn the case of pivotal tuning, [tools/batch_bundle_convert.py](tools/batch_bundle_convert.py) can be further used to convert to and from bundle formats.\nCheck [docs/Conversion-scripts.md](docs/Conversion-scripts.md) for more information.\n\n#### As standalone wrappers\n\nSee [example/standalone_example.py](example/standalone_example.py) for full example.\n\nImport `create_lycoris` and `LycorisNetwork` from `lycoris` library, put your preset to `LycorisNetwork` and then use `create_lycoris` to create LyCORIS module for your pytorch module.\n\nFor example:\n\n```py\nfrom lycoris import create_lycoris, LycorisNetwork\n\nLycorisNetwork.apply_preset(\n    {\"target_name\": [\".*attn.*\"]}\n)\nlycoris_net = create_lycoris(\n    your_model, \n    1.0, \n    linear_dim=16, \n    linear_alpha=2.0, \n    algo=\"lokr\"\n)\nlycoris_net.apply_to()\n\n# after apply_to(), your_model() will run with LyCORIS net\nlycoris_param = lycoris_net.parameters()\nforward_with_lyco = your_model(x)\n```\n\nYou can check my [HakuPhi](https://github.com/KohakuBlueleaf/HakuPhi) project to see how I utilize LyCORIS to finetune the Phi-1.5 models.\n\n#### Other method\n\nAfter LyCORIS3.0.0, Parametrize API and Functional API have been added, which provide more different ways on utilizing LyCORIS library.\n\nCheck API reference for more information.\nYou can also take the [test suites](test/) as a kind of examples.\n\n#### Bitsandbytes support\n\nSee [example/bnb_example.py](example/bnb_example.py) for example. Basically as same as standalone wrapper.\n\n#### Graphical interfaces and Colabs (via kohya trainer)\n\nYou can also train LyCORIS with the following graphical interfaces\n\n* [bmaltais/kohya_ss](https://github.com/bmaltais/kohya_ss)\n* [derrian-distro/LoRA_Easy_Training_Scripts](https://github.com/derrian-distro/LoRA_Easy_Training_Scripts)\n* [Akegarasu/lora-scripts](https://github.com/Akegarasu/lora-scripts)\n\nand colabs (please help us complete the list!)\n\n* [hollowstrawberry/kohya-colab](https://github.com/hollowstrawberry/kohya-colab)\n* [Linaqruf/kohya-trainer](https://github.com/Linaqruf/kohya-trainer)\n\nHowever, they are not guaranteed to be up-to-date. In particular, newer types may not be supported. Consider requesting the developers for support or simply use the original kohya script in this case.\n\n## Utilities\n\n### Extract LoCon\n\nYou can extract LoCon from a dreambooth model with its base model.\n\n```bash\npython3 extract_locon.py \u003csettings\u003e \u003cbase_model\u003e \u003cdb_model\u003e \u003coutput\u003e\n```\n\nUse --help to get more info\n\n```\n$ python3 extract_locon.py --help\nusage: extract_locon.py [-h] [--is_v2] [--is_sdxl] [--device DEVICE] [--mode MODE] [--safetensors] [--linear_dim LINEAR_DIM]\n                        [--conv_dim CONV_DIM] [--linear_threshold LINEAR_THRESHOLD] [--conv_threshold CONV_THRESHOLD]\n                        [--linear_ratio LINEAR_RATIO] [--conv_ratio CONV_RATIO] [--linear_quantile LINEAR_QUANTILE]\n                        [--conv_quantile CONV_QUANTILE] [--use_sparse_bias] [--sparsity SPARSITY] [--disable_cp]\n                        base_model db_model output_name\n```\n\n### Merge LyCORIS back to model\n\nYou can merge your LyCORIS model back to your checkpoint (base model).\n\n```bash\npython3 merge.py \u003csettings\u003e \u003cbase_model\u003e \u003clycoris_model\u003e \u003coutput\u003e\n```\n\nUse --help to get more info\n\n```\n$ python3 merge.py --help\nusage: merge.py [-h] [--is_v2] [--is_sdxl] [--device DEVICE] [--dtype DTYPE] [--weight WEIGHT] base_model lycoris_model output_name\n```\n\n### Conversion of LoRA, LyCORIS and full models between HCP and sd-webui format\n\nThis script allows you to use the LyCORIS models trained with HCP-Diffusion in sd-webui.\n\n```bash\npython3 batch_hcp_convert.py \\\n  --network_path /path/to/ckpts \\\n  --dst_dir /path/to/stable-diffusion-webui/models/Lora \\\n  --output_prefix something \\\n  --auto_scale_alpha --to_webui\n```\n\nSee [docs/Conversion-scripts.md](docs/Conversion-scripts.md) for more information.\n\n### Conversion from and to bundle format\n\nThis script is particularly useful in the case of pivotal tuning.\n\n```bash\npython3 batch_bundle_convert.py \\\n  --network_path /path/to/sd-webui-ssd/models/Lora  \\\n  --emb_path /path/to/ckpts \\\n  --dst_dir /path/to/sd-webui-ssd/models/Lora/bundle \\\n  --to_bundle --verbose 2 \n```\n\nSee [docs/Conversion-scripts.md](docs/Conversion-scripts.md) for more information.\n\n## Change Log\n\nFor full log, please see [Change.md](Change.md)\n\n### 2025/04/23 update to 3.2.0\n\n#### New Features\n\n* Support lora-plus learning rate scaling\n* Support HunYuanVideo model and Wan2.1 model\n* LyCORIS now have `onfly_merge` and `onfly_restore` method. Which can be used in inference time to merge the weights of LyCORIS into the original model. This will save the memory and speed up the inference time.\n\n#### Improvements\n\n* [BREAKING CHANGES] Now LyCORIS will use `wd_on_output=True` by default. This will make the weight norm more consistent with the original paper.\n\n#### Bug fixes\n\n* `bypass_mode=False` will turn off the bypass mode correctly now.\n\n## Todo list\n\n- [ ] Automatically selecting an algorithm based on the specific rank requirement.\n- [ ] More experiments for different task, not only diffusion models.\n  - [X] LoKr and LoHa have been proven to be useful for Large Language Model.\n- [ ] Explore other low-rank representations or parameter-efficient methods to fine-tune either the entire model or specific parts of it.\n- [ ] Documentation for whole library.\n\n## Citation\n\n```bibtex\n@inproceedings{\n  yeh2024navigating,\n  title={Navigating Text-To-Image Customization: From Ly{CORIS} Fine-Tuning to Model Evaluation},\n  author={SHIH-YING YEH and Yu-Guan Hsieh and Zhidong Gao and Bernard B W Yang and Giyeong Oh and Yanmin Gong},\n  booktitle={The Twelfth International Conference on Learning Representations},\n  year={2024},\n  url={https://openreview.net/forum?id=wfzXa8e783}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKohakuBlueleaf%2FLyCORIS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKohakuBlueleaf%2FLyCORIS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKohakuBlueleaf%2FLyCORIS/lists"}