{"id":20801137,"url":"https://github.com/gcorso/particle-guidance","last_synced_at":"2025-06-30T05:04:58.578Z","repository":{"id":201062293,"uuid":"706874837","full_name":"gcorso/particle-guidance","owner":"gcorso","description":"Implementation of Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models","archived":false,"fork":false,"pushed_at":"2023-10-23T13:26:59.000Z","size":11667,"stargazers_count":67,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-07T00:11:40.486Z","etag":null,"topics":["diffusion-models","generative-model","image-generation","stable-diffusion"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2310.13102","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gcorso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-18T19:35:24.000Z","updated_at":"2025-02-24T09:40:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"a211aec1-b8a3-4e5a-9e1f-c69309faee7e","html_url":"https://github.com/gcorso/particle-guidance","commit_stats":null,"previous_names":["gcorso/particle-guidance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gcorso/particle-guidance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gcorso%2Fparticle-guidance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gcorso%2Fparticle-guidance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gcorso%2Fparticle-guidance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gcorso%2Fparticle-guidance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gcorso","download_url":"https://codeload.github.com/gcorso/particle-guidance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gcorso%2Fparticle-guidance/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262714506,"owners_count":23352464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion-models","generative-model","image-generation","stable-diffusion"],"created_at":"2024-11-17T18:16:51.346Z","updated_at":"2025-06-30T05:04:58.541Z","avatar_url":"https://github.com/gcorso.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Particle Guidance \n\nImplementation of [Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models](https://arxiv.org/abs/2310.13102) by Gabriele Corso, Yilun Xu, Valentin de Bortoli, Regina Barzilay and Tommi Jaakkola.\n\nGenerative models are often sampled multiple times to obtain a diverse set incurring in a cost that is orthogonal to sampling time. To improve diversity and sample efficiency we need to move beyond the common assumption of independent samples. Particle guidance is an extension of diffusion-based generative sampling where a joint-particle time-evolving potential enforces diversity. In our manuscript analyze theoretically the joint distribution that particle guidance generates, its implications on the choice of potential, and the connections with methods in other disciplines. Here you can find the code for our empirical experiments. We test the framework both in the setting of conditional image generation, where we are able to increase diversity without affecting quality, and molecular conformer generation, where we reduce the state-of-the-art median error by 13% on average.\n\nIf you have questions, don't hesitate to open an issue or send us an email at gcorso@mit.edu and ylxu@mit.edu.\n\n![i.i.d](stable_diffusion/assets/main_figure.png)\n\n## Synthetic experiments\n\nFor our synthetic experiments on mixture of Gaussians distribution see the `synthetic.ipynb` notebook.\n\n## Stable Diffusion\n\nYou can find all the code in the `stable_diffusion` folder.\n\n### Running visualization\n\nThe code depends on PyTorch \u003e= 1.2, numpy, and matplotlib. It has been tested on both Python 3.9. Please make sure you're logged in with `huggingface-cli login` , in order to download the Stable Diffusion-1.5 model.\n\n|                   I.I.D.                    |                  Particle Guidance (pixel)                   |                 Particle Guidance (feature)                  |\n|:-------------------------------------------:| :----------------------------------------------------------: | :----------------------------------------------------------: |\n| ![i.i.d](stable_diffusion/assets/i.i.d.png) | ![pixel](stable_diffusion/assets/pixel.png) | ![dino](stable_diffusion/assets/dino.png) |\n\nWe've integrated the (1) vanilla i.i.d. sampling; (2) Particle Guidance (pixel); (3) Particle Guidance (feature) in the new pipeline `./diffuser/pipelines/stable_diffusion/pipeline_stable_diffusion_particle.py`\n\n**See `particle_guidance_demo.ipynb` for the procedures to produce the visualizations in the paper.** Feel free to try your own prompts :rocket:\n\n\n\n### Quantitative comparison\n\n#### 1. Data processing:\n\n- Step 1: Follow the instruction on top of `data_process.py`\n- Step 2: Run `python3 data_process.py` to randomly sampled 5K image-text pair from COCO validation set.\n\n#### 2. Generation\n\n```sh\ntorchrun --rdzv_endpoint=0.0.0.0:1201 --nproc_per_node=8 generate_particle.py \n\t--save_path: path to save images\n\t--w: classifier-guidance weight \n\t--name: name of experiments\n\t--dino: doing particle guidance in the DINO feature space\n\t--coeff: coefficient of the particle guidance term (0: i.i.d.)\n```\n\n#### 3. Evaluation (Aesthetic score, CLIP score,  in-batch similarity score)\n\n```shell\npython3 eval_clip_score_particle.py  --dir_path {path}\n\npath: path to the directory of generated image\n```\n\n\n\n## Molecular Conformer Generation \n\nYou can find the code for running Particle Guidance on Torsional Diffusion directly integrated in the [torsional diffusion code repository](https://github.com/gcorso/torsional-diffusion). To run the particle guidance sampling with torsional diffusion to replicate the results of the paper (similarly you can run on your own molecules).\n\nFor the permutation invariant kernel guidance (higher quality, slower):\n\n    # minimizing recall error\n    python generate_confs.py --tqdm --batch_size 128 --no_energy --inference_steps=20 --model_dir=workdir/drugs_default --test_csv=data/DRUGS/test_smiles.csv --pg_invariant=True --pg_kernel_size_log_0=1.7565691770646286 --pg_kernel_size_log_1=1.1960868735428605 --pg_langevin_weight_log_0=-2.2245183818892103 --pg_langevin_weight_log_1=-2.403905082248579 --pg_repulsive_weight_log_0=-2.158537381110402 --pg_repulsive_weight_log_1=-2.717482077162461 --pg_weight_log_0=0.8004013644746992 --pg_weight_log_1=-0.9255658381081596\n    # minimizing precision error\n    python generate_confs.py --tqdm --batch_size 128 --no_energy --inference_steps=20 --model_dir=workdir/drugs_default --test_csv=data/DRUGS/test_smiles.csv --pg_invariant=True --pg_kernel_size_log_0=-0.9686202580381296 --pg_kernel_size_log_1=-0.7808409291022302 --pg_langevin_weight_log_0=-2.434216242826782 --pg_langevin_weight_log_1=-0.2602238633333869 --pg_repulsive_weight_log_0=-2.0439285313973237 --pg_repulsive_weight_log_1=-1.468234554877924 --pg_weight_log_0=0.3495680598729498 --pg_weight_log_1=-0.22001939454654185\n\n\nFor the non-permutation invariant kernel guidance (faster, slightly lower quality, but still better than I.I.D.):\n\n    # minimizing recall error\n    python generate_confs.py --tqdm --batch_size 128 --no_energy --inference_steps=20 --model_dir=workdir/drugs_default --test_csv=data/DRUGS/test_smiles.csv --pg_kernel_size_log_0=2.35958 --pg_kernel_size_log_1=-0.78826 --pg_langevin_weight_log_0=-1.55054 --pg_langevin_weight_log_1=-2.70316 --pg_repulsive_weight_log_0=1.01317 --pg_repulsive_weight_log_1=-2.68407 --pg_weight_log_0=0.60504 --pg_weight_log_1=-1.15020\n    # minimizing precision error\n    python generate_confs.py --tqdm --batch_size 128 --no_energy --inference_steps=20 --model_dir=workdir/drugs_default --test_csv=data/DRUGS/test_smiles.csv --pg_kernel_size_log_0=1.29503 --pg_kernel_size_log_1=1.45944 --pg_langevin_weight_log_0=-2.88867 --pg_langevin_weight_log_1=-2.47591 --pg_repulsive_weight_log_0=-1.01222 --pg_repulsive_weight_log_1=-1.91253 --pg_weight_log_0=-0.16253 --pg_weight_log_1=0.79355\n\n\n\n## Citation\n    @article{corso2023particle,\n          title={Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models}, \n          author = {Corso, Gabriele and Xu, Yilun and de Bortoli, Valentin and Barzilay, Regina and Jaakkola, Tommi},\n          journal={arXiv},\n          year={2023}\n    }\n\n## License\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgcorso%2Fparticle-guidance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgcorso%2Fparticle-guidance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgcorso%2Fparticle-guidance/lists"}