{"id":18764440,"url":"https://github.com/tripplyons/sd-ia3","last_synced_at":"2025-04-13T04:33:28.790Z","repository":{"id":149996634,"uuid":"612048964","full_name":"tripplyons/sd-ia3","owner":"tripplyons","description":"(IA)^3 for Stable Diffusion","archived":false,"fork":false,"pushed_at":"2023-04-02T19:43:30.000Z","size":2189,"stargazers_count":35,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-26T21:47:16.722Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tripplyons.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-10T04:47:30.000Z","updated_at":"2024-12-22T22:20:08.000Z","dependencies_parsed_at":"2023-04-19T16:47:15.567Z","dependency_job_id":null,"html_url":"https://github.com/tripplyons/sd-ia3","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tripplyons%2Fsd-ia3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tripplyons%2Fsd-ia3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tripplyons%2Fsd-ia3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tripplyons%2Fsd-ia3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tripplyons","download_url":"https://codeload.github.com/tripplyons/sd-ia3/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248664666,"owners_count":21142005,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T18:29:52.412Z","updated_at":"2025-04-13T04:33:28.785Z","avatar_url":"https://github.com/tripplyons.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# (IA)^3 for Stable Diffusion\n\nParameter-efficient fine-tuning of Stable Diffusion using (IA)^3.\n\n## YouTube Video Explanation\n\n[![YouTube Video Explanation](https://img.youtube.com/vi/M5gjAthTwho/0.jpg)](https://www.youtube.com/watch?v=M5gjAthTwho)\n\n## Example\n\n| Before Fine-Tuning | After Fine-Tuning |\n| --- | --- |\n| ![Donald Trump](/docs/no-adapter-example.png) | ![Donald Trump](/docs/adapter-example.png) |\n\nThe prompt is \"donald trump\", and the model is fine-tuned on [pokemon-blip-captions](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions) for 25 epochs.\n\n## Description\n\nBased on these papers:\n\n- (IA)^3: [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://arxiv.org/abs/2205.05638)\n- Stable Diffusion: [High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)\n\nImplemented in [diffusers](https://github.com/huggingface/diffusers) using an attention processor in [`attention.py`](/attention.py).\n\n## Comparison to full fine-tuning\n\n(IA)^3 has trade-offs similar to LoRA when comparing to full fine-tuning.\n\nOne major difference to LoRA is that (IA)^3 uses much less parameters. In general, it will most likely be faster and smaller, but less expressive.\n\n- Faster training\n- Smaller file size (~222 KB for Stable Diffusion 1.5 when `learn_biases=False`, about twice as much otherwise)\n- Can be swapped in and out of the base model during inference\n- Can be loaded into fine-tuned models that have the same architecture\n- Can be merged with the weights of the base model\n  - Only possible when `learn_biases=False` without changing the architecture\n  - Not currently implemented in this repo\n\n## Installation\n\nFirst create an environment and [install PyTorch](https://pytorch.org/get-started/locally/).\n\nThen install the pip dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\nCurrently, bitsandbytes only supports Linux, so fine-tuning on Windows requires more VRAM.\n\n## Training\n\nTraining script in [`train.py`](/train.py). Based on [this example script for diffusers](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py).\n\nCurrently you can change the parameters by editing the variables at the top of the file and running the script:\n\n```bash\npython train.py\n```\n\n## Inference\n\nInference script in [`infer.py`](/infer.py) to load the changes and generate images.\n\nCurrently you can change the parameters by editing the variables at the top of the file and running the script:\n\n```bash\npython infer.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftripplyons%2Fsd-ia3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftripplyons%2Fsd-ia3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftripplyons%2Fsd-ia3/lists"}