{"id":13561320,"url":"https://github.com/cloneofsimo/d3pm","last_synced_at":"2025-04-13T09:37:57.072Z","repository":{"id":233258201,"uuid":"785087480","full_name":"cloneofsimo/d3pm","owner":"cloneofsimo","description":"Minimal Implementation of a D3PM in pytorch","archived":false,"fork":false,"pushed_at":"2024-04-22T14:11:50.000Z","size":1985,"stargazers_count":212,"open_issues_count":6,"forks_count":16,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-11T18:54:35.146Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://proceedings.neurips.cc/paper/2021/hash/958c530554f78bcd8e97125b70e6973d-Abstract.html","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloneofsimo.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-11T07:04:56.000Z","updated_at":"2025-04-10T03:03:43.000Z","dependencies_parsed_at":"2024-08-01T13:17:00.027Z","dependency_job_id":"9ecebcae-c7c7-4294-aab5-1d7be4679c84","html_url":"https://github.com/cloneofsimo/d3pm","commit_stats":null,"previous_names":["cloneofsimo/d3pm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fd3pm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fd3pm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fd3pm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloneofsimo%2Fd3pm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloneofsimo","download_url":"https://codeload.github.com/cloneofsimo/d3pm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248691768,"owners_count":21146437,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:00:54.851Z","updated_at":"2025-04-13T09:37:57.051Z","avatar_url":"https://github.com/cloneofsimo.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"contents/output.gif\" alt=\"large\" width=\"400\"\u003e\n  \u003cimg src=\"contents/cifar_best.gif\" alt=\"large\" width=\"200\"\u003e\n\u003c/p\u003e\n\n\n# Minimal Implementation of a D3PM (Structured Denoising Diffusion Models in Discrete State-Spaces), in pytorch\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"contents/best.gif\" alt=\"small\" width=\"400\"\u003e\n  \u003cimg src=\"contents/best.png\" alt=\"small\" width=\"400\"\u003e\n\u003c/p\u003e\n\n\n**Special thanks to [fal.ai](https://fal.ai/) for the compute resources for this project.**\n\n\nThis is minimal (400 LOC), but fully faithful implementation of a D3PM [Structured Denoising Diffusion Models in Discrete State-Spaces](https://arxiv.org/abs/2107.03006). in pytorch.\n\nI have tried to keep the code as simple as possible with much comments and explanation that is somewhat lacking on the original jax implementation, so that it is easy to understand. As far as I know, this is the first, faithful reimplementation of D3PM in pytorch. (Please correct me if I am wrong). Of course, this implementation was heavily based on the [official implementation](https://github.com/google-research/google-research/tree/master/d3pm/images).\n\nDifference between this implementation and the official implementation:\n\n* This one has conditional sampling, so as you can see, generations are class-conditioned.\n* This one uses rather different/simple model architecture.\n* This one simplfies the official implementation very very much, so it is 400 LOC.\n* This one does not use truncated logistic reparameterization, but you can use that if you wish.\n* Only has uniform sample with inverse-linear beta scheudule, but you can change that with couple loc as well.\n\n## Usage\n\n\nFollowing is completely self-contained example.\n\n```bash\npython d3pm_runner.py\n```\n\nFollowing uses dit.py, for CIFAR-10 dataset.\n  \n```bash\npython d3pm_runner_cifar.py\n```\n\n## Requirements\n\nInstall torch, torchvision, pillow, tqdm\n\n```bash\npip install torch torchvision pillow tqdm\n```\n\n## Citation\n\nThis implementation:\n\n```bibtex\n@misc{d3pm_pytorch,\n  author={Simo Ryu},\n  title={Minimal Implementation of a D3PM (Structured Denoising Diffusion Models in Discrete State-Spaces), in pytorch},\n  year={2024},\n  howpublished={\\url{https://github.com/cloneofsimo/d3pm}}\n}\n```\n\nOriginal Paper:\n\n```bibtex\n@article{austin2021structured,\n  title={Structured denoising diffusion models in discrete state-spaces},\n  author={Austin, Jacob and Johnson, Daniel D and Ho, Jonathan and Tarlow, Daniel and Van Den Berg, Rianne},\n  journal={Advances in Neural Information Processing Systems},\n  volume={34},\n  pages={17981--17993},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloneofsimo%2Fd3pm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloneofsimo%2Fd3pm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloneofsimo%2Fd3pm/lists"}