{"id":13788138,"url":"https://github.com/castorini/daam","last_synced_at":"2025-05-16T06:05:45.895Z","repository":{"id":63060175,"uuid":"548093252","full_name":"castorini/daam","owner":"castorini","description":"Diffusion attentive attribution maps for interpreting Stable Diffusion.","archived":false,"fork":false,"pushed_at":"2024-04-05T20:02:28.000Z","size":2251,"stargazers_count":754,"open_issues_count":15,"forks_count":65,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-09T12:23:57.070Z","etag":null,"topics":["diffusion","explainable-ai","generative-ai","huggingface","pytorch","stable-diffusion"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/castorini.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-08T23:02:12.000Z","updated_at":"2025-05-08T08:46:26.000Z","dependencies_parsed_at":"2023-11-30T01:27:41.323Z","dependency_job_id":"c22f769c-4cd3-412a-ab56-d42e06958dd7","html_url":"https://github.com/castorini/daam","commit_stats":{"total_commits":81,"total_committers":8,"mean_commits":10.125,"dds":"0.37037037037037035","last_synced_commit":"119d8ff1dd4e61ef579824f3112fb0010eb2fff0"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Fdaam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Fdaam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Fdaam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/castorini%2Fdaam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/castorini","download_url":"https://codeload.github.com/castorini/daam/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478188,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion","explainable-ai","generative-ai","huggingface","pytorch","stable-diffusion"],"created_at":"2024-08-03T21:00:37.364Z","updated_at":"2025-05-16T06:05:44.424Z","avatar_url":"https://github.com/castorini.png","language":"Jupyter Notebook","funding_links":[],"categories":["Complementary Models and Tools"],"sub_categories":["Potentially Stale/Less active branches"],"readme":"# What the DAAM: Interpreting Stable Diffusion Using Cross Attention\n\n[![HF Spaces](https://img.shields.io/badge/HuggingFace%20Space-online-green.svg)](https://huggingface.co/spaces/tetrisd/Diffusion-Attentive-Attribution-Maps) [![Citation](https://img.shields.io/badge/Citation-ACL-orange.svg)](https://gist.github.com/daemon/639de6fea584d7df1a62f04a2ea0cdad) [![PyPi version](https://badgen.net/pypi/v/daam?color=blue)](https://pypi.org/project/daam) [![Downloads](https://static.pepy.tech/badge/daam)](https://pepy.tech/project/daam)\n\n![example image](example.jpg)\n\n### Updated to support Stable Diffusion XL (SDXL) and Diffusers 0.21.1!\n\nI regularly update this codebase. Please submit an issue if you have any questions.\n\nIn [our paper](https://aclanthology.org/2023.acl-long.310), we propose diffusion attentive attribution maps (DAAM), a cross attention-based approach for interpreting Stable Diffusion.\nCheck out our demo: https://huggingface.co/spaces/tetrisd/Diffusion-Attentive-Attribution-Maps.\nSee our [documentation](https://castorini.github.io/daam/), hosted by GitHub pages, and [our Colab notebook](https://colab.research.google.com/drive/1miGauqa07uHnDoe81NmbmtTtnupmlipv?usp=sharing), updated for v0.1.0.\n\n## Getting Started\nFirst, install [PyTorch](https://pytorch.org) for your platform.\nThen, install DAAM with `pip install daam`, unless you want an editable version of the library, in which case do `git clone https://github.com/castorini/daam \u0026\u0026 pip install -e daam`.\nFinally, login using `huggingface-cli login` to get many stable diffusion models -- you'll need to get a token at [HuggingFace.co](https://huggingface.co/).\n\n### Running the Website Demo\nSimply run `daam-demo` in a shell and navigate to http://localhost:8080.\nThe same demo as the one on HuggingFace Spaces will show up.\n\n### Using DAAM as a CLI Utility\nDAAM comes with a simple generation script for people who want to quickly try it out.\nTry running\n```bash\n$ mkdir -p daam-test \u0026\u0026 cd daam-test\n$ daam \"A dog running across the field.\"\n$ ls\na.heat_map.png    field.heat_map.png    generation.pt   output.png  seed.txt\ndog.heat_map.png  running.heat_map.png  prompt.txt\n```\nYour current working directory will now contain the generated image as `output.png` and a DAAM map for every word, as well as some auxiliary data.\nYou can see more options for `daam` by running `daam -h`.\nTo use Stable Diffusion XL as the backend, run `daam --model xl-base-1.0 \"Dog jumping\"`.  \n\n### Using DAAM as a Library\n\nImport and use DAAM as follows:\n\n```python\nfrom daam import trace, set_seed\nfrom diffusers import DiffusionPipeline\nfrom matplotlib import pyplot as plt\nimport torch\n\n\nmodel_id = 'stabilityai/stable-diffusion-xl-base-1.0'\ndevice = 'cuda'\n\npipe = DiffusionPipeline.from_pretrained(model_id, use_auth_token=True, torch_dtype=torch.float16, use_safetensors=True, variant='fp16')\npipe = pipe.to(device)\n\nprompt = 'A dog runs across the field'\ngen = set_seed(0)  # for reproducibility\n\nwith torch.no_grad():\n    with trace(pipe) as tc:\n        out = pipe(prompt, num_inference_steps=50, generator=gen)\n        heat_map = tc.compute_global_heat_map()\n        heat_map = heat_map.compute_word_heat_map('dog')\n        heat_map.plot_overlay(out.images[0])\n        plt.show()\n```\n\nYou can also serialize and deserialize the DAAM maps pretty easily:\n\n```python\nfrom daam import GenerationExperiment, trace\n\nwith trace(pipe) as tc:\n    pipe('A dog and a cat')\n    exp = tc.to_experiment('experiment-dir')\n    exp.save()  # experiment-dir now contains all the data and heat maps\n\nexp = GenerationExperiment.load('experiment-dir')  # load the experiment\n```\n\nWe'll continue adding docs.\nIn the meantime, check out the `GenerationExperiment`, `GlobalHeatMap`, and `DiffusionHeatMapHooker` classes, as well as the `daam/run/*.py` example scripts.\nYou can download the COCO-Gen dataset from the paper at http://ralphtang.com/coco-gen.tar.gz.\nIf clicking the link doesn't work on your browser, copy and paste it in a new tab, or use a CLI utility such as `wget`.\n\n## See Also\n- [DAAM-i2i](https://github.com/RishiDarkDevil/daam-i2i), an extension of DAAM to image-to-image attribution.\n\n- [Furkan's video](https://www.youtube.com/watch?v=XiKyEKJrTLQ) on easily getting started with DAAM.\n\n- [1littlecoder's video](https://www.youtube.com/watch?v=J2WtkA1Xfew) for a code demonstration and Colab notebook of an older version of DAAM.\n\n## Citation\n```\n@inproceedings{tang2023daam,\n    title = \"What the {DAAM}: Interpreting Stable Diffusion Using Cross Attention\",\n    author = \"Tang, Raphael  and\n      Liu, Linqing  and\n      Pandey, Akshat  and\n      Jiang, Zhiying  and\n      Yang, Gefei  and\n      Kumar, Karun  and\n      Stenetorp, Pontus  and\n      Lin, Jimmy  and\n      Ture, Ferhan\",\n    booktitle = \"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\",\n    year = \"2023\",\n    url = \"https://aclanthology.org/2023.acl-long.310\",\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcastorini%2Fdaam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcastorini%2Fdaam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcastorini%2Fdaam/lists"}