{"id":23739858,"url":"https://github.com/p1atdev/leco","last_synced_at":"2025-04-06T21:16:25.512Z","repository":{"id":176875964,"uuid":"656648289","full_name":"p1atdev/LECO","owner":"p1atdev","description":"Low-rank adaptation for Erasing COncepts from diffusion models.","archived":false,"fork":false,"pushed_at":"2024-01-20T13:51:10.000Z","size":4005,"stargazers_count":322,"open_issues_count":22,"forks_count":27,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-30T19:08:30.341Z","etag":null,"topics":["diffusion","lora","stable-diffusion"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2303.07345","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/p1atdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-21T11:04:11.000Z","updated_at":"2025-03-22T15:34:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"1120cfd4-db82-41d2-bc8e-21b16c1c0f48","html_url":"https://github.com/p1atdev/LECO","commit_stats":null,"previous_names":["p1atdev/leco"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p1atdev%2FLECO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p1atdev%2FLECO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p1atdev%2FLECO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p1atdev%2FLECO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/p1atdev","download_url":"https://codeload.github.com/p1atdev/LECO/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247550691,"owners_count":20956987,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion","lora","stable-diffusion"],"created_at":"2024-12-31T09:39:34.341Z","updated_at":"2025-04-06T21:16:25.493Z","avatar_url":"https://github.com/p1atdev.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LECO ✏️ \n\n\u003ca href=\"https://colab.research.google.com/github/p1atdev/LECO/blob/main/train.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\nLow-rank adaptation for Erasing COncepts from diffusion models.  \n\nThe original repository: [Erasing Concepts from Diffusion Models](https://github.com/rohitgandikota/erasing/tree/main)\n\nand the project page: https://erasing.baulab.info/\n\n(Not only for erasing concepts, but also emphasizing or swapping them by devising prompts and LoRA weight. See [ConceptMod](https://github.com/ntc-ai/conceptmod) for more details)\n\n## Setup\n\n```bash\nconda create -n leco python=3.10\npip install torch torchvision --index-url https://download.pytorch.org/whl/cu118\npip install xformers\npip install -r requirements.txt\n```\n\n## Train\n\nYou need 8GB VRAM at least.\n\n```bash\npython ./train_lora.py --config_file \"./examples/config.yaml\"\n```\n\n`config.yaml`:\n\n```yaml\nprompts_file: \"./prompts.yaml\"\n\npretrained_model:\n  name_or_path: \"stabilityai/stable-diffusion-2-1\" # you can also use .ckpt or .safetensors models\n  v2: true # true if model is v2.x\n  v_pred: true # true if model uses v-prediction\n\nnetwork:\n  type: \"lierla\" # or \"c3lier\"\n  rank: 4\n  alpha: 1.0\n\ntrain:\n  precision: \"bfloat16\"\n  noise_scheduler: \"ddim\" # or \"ddpm\", \"lms\", \"euler_a\"\n  iterations: 500\n  lr: 1e-4\n  optimizer: \"AdamW\"\n  lr_scheduler: \"constant\"\n\nsave:\n  name: \"van_gogh\"\n  path: \"./output\"\n  per_steps: 200\n  precision: \"bfloat16\"\n\nlogging:\n  use_wandb: false\n  verbose: false\n\nother:\n  use_xformers: true\n```\n\n`prompts.yaml`:\n\n```yaml\n- target: \"van gogh\" # what word for erasing the positive concept from\n  positive: \"van gogh\" # concept to erase\n  unconditional: \"\" # word to take the difference from the positive concept\n  neutral: \"\" # starting point for conditioning the target\n  action: \"erase\" # erase or enhance\n  guidance_scale: 1.0\n  resolution: 512\n  dynamic_resolution: false\n  batch_size: 2\n```\n\nSee the [example config](/examples/config.yaml) for more details.\n\nNote: You can use float16 but it is unstable and not recommended. Please use bfloat16 or float32. \n\n## Pretrained weights\n\nYou can use the pretrained weights on AUTOMATIC1111's webui. \n\n🤗 HuggingFace: https://huggingface.co/p1atdev/leco\n\n### SDv1.5\n\n- [Van Gogh style](https://huggingface.co/p1atdev/leco/blob/main/van_gogh_sdv15.safetensors) (trained to erase the concept of \"van gogh style\" on SDv1.5)\n\nResults of `oil painting of van gogh by himself`:\n\n![van gogh](./images/van_gogh.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\noil painting of van gogh by himself\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3870472781, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\nResults of `painting of scenery by monet`:\n\n![monet](./images/van_gogh_monet.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\npainting of scenery by monet\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1284787312, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: van_gogh_4_last(db68853d039b), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\n### SDv2.1-768\n\n- [Mona Lisa](https://huggingface.co/p1atdev/leco/blob/main/mona_lisa_sdv21v.safetensors) (trained to erase the concept of \"mona lisa\" on SDv2.1-768)\n\n\nResults of `mona lisa with jewelry`:\n\n![mona lisa](./images/mona_lisa.jpg)\n\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\nmona lisa with jewelry\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 3630495347, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\nResults of `photo of a cute cat`:\n\n![mona lisa](./images/mona_lisa_cat.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\nphoto of a cute cat\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 900866192, Size: 512x512, Model hash: 832eb50c0c, Model: v2-1_768-ema-pruned, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: mona_lisa2_last(393beb35c4b1), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\n### WD1.5 beta3\n\n- [Cat ears](https://huggingface.co/p1atdev/leco/blob/main/cat_ears_wd15beta3.safetensors) (trained to replace \"1girl\" with \"1girl, cat ears\" on WD1.5 beta3 )\n\nCat ears will be attached forcibly when using with 1.0~3.0 weight. \n\nIf -1.0~-3.0, cat ears will never appear.\n\nTraining settings: see [configs](./examples/cat_ears_config.yaml).\n\n![cat ears](./images/cat_ears.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\nmasterpiece, best quality, exceptional, best aesthetic, anime, 1girl, school uniform, upper body, smile\nNegative prompt: worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 4103955758, Size: 512x512, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"0, 2, 3, 4\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\n\n- [Unreal](https://huggingface.co/p1atdev/leco/blob/main/unreal_wd15beta3.safetensors) (trained to erase \"realistic\", \"real life\", \"instagram\" on WD1.5 beta3)\n\nTraining settings: see [configs](./examples/unreal_config.yaml).\n\nWith \"real life, instagram\":\n\n![unreal](./images/unreal_blue_cat.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\nreal life, instagram, masterpiece, best quality, exceptional, best aesthetic, 1girl, cat ears, blue hair, school uniform, upper body\nNegative prompt: worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 757542759, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\nWithout \"real life, instagram\":\n\n![unreal](./images/unreal_yellow_girl.jpg)\n\n\u003cdetails\u003e\n\u003csummary\u003e\nGeneration settings\n\u003c/summary\u003e\n\n```yaml\nmasterpiece, best quality, exceptional, best aesthetic,, 1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt,\nNegative prompt: worst quality, low quality, bad aesthetic, oldest, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry\nSteps: 20, Sampler: Euler a, CFG scale: 7, Seed: 2867636749, Size: 768x768, Model hash: d38e779546, Model: wd-beta3-base-fp16, Clip skip: 2, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: unreal_6_many_prompts_200steps(fff5917285da), AddNet Weight A 1: -1.0, AddNet Weight B 1: -1.0, Script: X/Y/Z plot, X Type: AddNet Weight 1, X Values: \"-1, 0, 1\", Version: v1.3.0\n```\n\n\u003c/details\u003e\n\n## References\n\nI am deeply inspired by and my work relies on the outstanding efforts of the following projects. I want to express my profound gratitude to these projects and their developers:\n\n- https://github.com/rohitgandikota/erasing: Erasing Concepts from Diffusion Models \n  - https://huggingface.co/spaces/baulab/Erasing-Concepts-In-Diffusion: Demo of ESD \n\n- https://github.com/cloneofsimo/lora: Low-rank Adaptation for Fast Text-to-Image Diffusion Fine-tuning\n\n- https://github.com/kohya-ss/sd-scripts: Training, generation and utility scripts for Stable Diffusion\n\n- https://github.com/ntc-ai/conceptmod:  Modify Concepts from Diffusion Models using a dsl \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fp1atdev%2Fleco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fp1atdev%2Fleco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fp1atdev%2Fleco/lists"}