{"id":44134392,"url":"https://gmum.github.io/DIAMOND/","last_synced_at":"2026-02-20T22:00:55.707Z","repository":{"id":336211025,"uuid":"1146051568","full_name":"gmum/DIAMOND","owner":"gmum","description":null,"archived":false,"fork":false,"pushed_at":"2026-02-10T22:37:20.000Z","size":787774,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-11T01:38:21.577Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gmum.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-30T15:03:15.000Z","updated_at":"2026-02-10T22:37:23.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/gmum/DIAMOND","commit_stats":null,"previous_names":["gmum/diamond"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gmum/DIAMOND","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmum%2FDIAMOND","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmum%2FDIAMOND/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmum%2FDIAMOND/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmum%2FDIAMOND/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gmum","download_url":"https://codeload.github.com/gmum/DIAMOND/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gmum%2FDIAMOND/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29666419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T19:49:36.704Z","status":"ssl_error","status_checked_at":"2026-02-20T19:44:05.372Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-08T23:00:22.912Z","updated_at":"2026-02-20T22:00:55.695Z","avatar_url":"https://github.com/gmum.png","language":"Python","funding_links":[],"categories":["3D-assets"],"sub_categories":[],"readme":"# DIAMOND: Directed Inference for Artifact Mitigation in Flow Matching Models\n\n\u003cdiv align=\"center\"\u003e\n\n🌐 **[Project Page](https://gmum.github.io/DIAMOND/)** \u0026nbsp;\u0026nbsp;|\u0026nbsp;\u0026nbsp; 📄 **[arXiv](https://arxiv.org/abs/2602.00883)**\n\u003cbr\u003e\n\n[Alicja Polowczyk*](https://www.linkedin.com/in/alicja-polowczyk-064739266/), [Agnieszka Polowczyk*](https://www.linkedin.com/in/agnieszka-polowczyk-91381323a/), [Piotr Borycki](https://www.linkedin.com/in/piotr-borycki-560052251), [Joanna Waczyńska](https://www.linkedin.com/in/joannawaczynska/), [Jacek Tabor](https://scholar.google.pl/citations?user=zSKYziUAAAAJ\u0026hl=pl), [Przemysław Spurek](https://scholar.google.com/citations?hl=en\u0026user=0kp0MbgAAAAJ)  \n(*equal contribution)\n\n\n\u003c/div\u003e\n\n---\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/teaser.jpg\" width=\"92%\"\u003e\n\u003c/p\u003e\n\n**DIAMOND** is a *training-free, inference-time guidance framework* that tackles one of the most persistent challenges in modern text-to-image generation: **visual and anatomical artifacts**.\n\nWhile recent models such as FLUX achieve impressive realism, they still frequently produce distorted structures, malformed anatomy, and visual inconsistencies. Unlike existing post-hoc or weight-modifying approaches, DIAMOND intervenes **directly during the generative process** by reconstructing a clean sample estimate at each step and **steering the sampling trajectory away from artifact-prone latent states**.\n\nThe method requires **no additional training, no finetuning, and no weight modification**, and can be applied to both **flow matching models and standard diffusion models**, enabling robust, zero-shot, high-fidelity image synthesis with substantially reduced artifacts.\n\n---\n\n## 📰 News\n\n- **Feb. 2026**: Initial codebase released with support for **FLUX models** (FLUX.1-dev, FLUX-schnell, FLUX-2-dev).\n- **Feb. 2026**: Paper is available on arXiv.\n- **Coming Soon**: **SDXL code** will be added to the repository.\n\n\n## ⚙️ Environment Setup\n\nWe provide two separate environment configurations depending on the model variant.\n\n### 🔹 Option A — FLUX.1 [dev], FLUX.1 [schnell], SDXL\n\n![Python](https://img.shields.io/badge/Python-3.11-blue)\n![PyTorch](https://img.shields.io/badge/PyTorch-2.6.0-red)\n![TorchVision](https://img.shields.io/badge/torchvision-0.21.0-orange)\n![Diffusers](https://img.shields.io/badge/diffusers-0.33.1-yellow)\n\nCreate and activate the Conda environment:\n\n```bash\nconda create -n diamond python=3.11 -y\nconda activate diamond\n```\nInstall PyTorch and remaining dependencies:\n```bash\npip install torch==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu126\npip install -r requirements.txt\n```\n\n### 🔹 Option B — FLUX-2-dev\nRequires a newer version of diffusers installed directly from GitHub.\n\n![Python](https://img.shields.io/badge/Python-3.10-blue)\n![PyTorch](https://img.shields.io/badge/PyTorch-2.5.1-red)\n![TorchVision](https://img.shields.io/badge/torchvision-0.20.1-orange)\n![TorchAudio](https://img.shields.io/badge/torchaudio-2.5.1-orange)\n![Diffusers](https://img.shields.io/badge/diffusers-github-yellow)\n\n```bash\nconda create -n diamond-flux2 python=3.10 -y\nconda activate diamond-flux2\n\npip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 \\\n  --index-url https://download.pytorch.org/whl/cu118\n\npip uninstall diffusers -y\npip install git+https://github.com/huggingface/diffusers.git -U\n\npip install -r requirements2.txt\n\n```\n## 📦 SOTA Method Weights\n\nWe release **our trained model weights** for several state-of-the-art artifact mitigation methods.\n\n\n| Base Model        | DiffDoctor | HPSv2 | HandsXL |\n|-----------------|------------|-------|---------|\n| FLUX.1 [dev]    | Coming Soon | Coming Soon | Coming Soon |\n| FLUX.1 [schnell]| Coming Soon | Coming Soon | — |\n| SDXL            | — | — | Coming Soon |\n| FLUX.2 [dev]   | — | — | — |\n\nFull evaluation datasets (CSV files with prompts and corresponding random seeds) are provided in the `datasets/` directory.  \nFor **SDXL**, a shortened dataset variant is released, as no random seeds producing artifact-containing images could be found for some prompts.\n\n# DIAMOND\n\n## 🚀 Generate a Single Image\n\nMove to the repository root:\n\n```bash\ncd DIAMOND\n```\nYou can select the base model using `model=dev` (**FLUX.1 [dev]**) or `model=schnell` (FLUX.1 **[schnell]**).\nSetting `guidance.enabled=true` enables **DIAMOND guidance** during sampling. To run **without DIAMOND (baseline)**, set `guidance.enabled=false`.\nYou can also modify the `loss` type and the `lambda_schedule` to explore different guidance behaviors.\n\n### Run Generation\n```bash\npython src/generate_single_image.py \\\n  model=dev \\\n  'prompt=\"Luxury crystal blue diamond, premium brand mark, vector style, simple and iconic, 4k resolution\"' \\\n  seed=100285 \\\n  guidance.enabled=false \\\n  loss=power \\\n  lambda_schedule=power \\\n  lambda_schedule.start=25 \\\n  lambda_schedule.end=1 \\\n  lambda_schedule.power=2 \\\n  output.run_name=example_run\n```\n\nFor **FLUX.2 [dev]**, use the separate script:\n```bash\npython src/generate_single_image_flux2.py \\\n  model=flux2dev \\\n  'prompt=\"Luxury crystal blue diamond, premium brand mark, vector style, simple and iconic, 4k resolution\"' \\\n  seed=100285 \\\n  output.run_name=example_run\n```\n\u003e [!IMPORTANT]\n\u003e Activate the correct Conda environment before running (see Environment Setup).\n\u003e Outputs are saved to the `outputs/` directory.\n\n### LoRA-based SOTA Methods\nSee the **📦 SOTA Method Weights** table for model support. Enable LoRA and set the appropriate checkpoint in `lora.path`.\n\n### Example (HandsXL)\n\n```bash\npython src/generate_single_image.py \\\n  model=dev \\\n  'prompt=\"A South Asian man, 35 years old, with a visual impairment, reading braille books in a library.\"' \\\n  seed=100283 \\\n  lora=enabled \\\n  lora.path=\"checkpoints/lora/people_handv1.safetensors\" \\\n  guidance.enabled=false \\\n  output.run_name=lora_example\n```\n\u003e [!IMPORTANT]\n\u003e When using LoRA-based SOTA methods, always set `guidance.enabled=false`.\n\n## 🚀 Generate Multiple Images\nThe generation setup is identical to single-image generation. **DIAMOND** can be enabled or disabled using `guidance.enabled=true/false`.  \n**LoRA-based SOTA** methods can be used by setting `lora=enabled` and specifying `lora.path`.\n \nFor **FLUX.1 [dev]**, **FLUX.1 [schnell]**, use:\n```bash\npython src/generate_images_csv.py \\\n  model=schnell \\\n  csv_path=/path/to/prompts.csv \\\n  loss=power \\\n  lambda_schedule=power \\\n  lambda_schedule.start=25 \\\n  lambda_schedule.end=1 \\\n  lambda_schedule.power=2 \\\n  output.run_name=example_run\n```\nFor **FLUX.2 [dev]**, use:\n```bash\npython src/generate_csv_flux2.py \\\n  model=flux2dev \\\n  csv_path=/path/to/prompts.csv \\\n  loss=power \\\n  lambda_schedule=power \\\n  lambda_schedule.start=25 \\\n  lambda_schedule.end=1 \\\n  lambda_schedule.power=2 \\\n  output.run_name=example_run\n```\n\n## 📊 Evaluation / Metrics\nThis script computes quantitative evaluation metrics for generated images.  \nResults are saved to `outputs/metrics/results.txt` by default and can be customized if needed.\n\nThe following metrics are computed: **CLIP-T**, **MeanArtifactFreq (%)**, **ArtifactPixelRatio (%)**, **MAE**, **MAE(A)**, **MAE(NA)**.\n\n#### Run metric computation:\n\n```bash\npython src/generate_metrics.py \\\n  metrics.generated_dir=/path/to/generated/images \\\n  metrics.reference_dir=/path/to/reference/images \\\n  metrics.prompts_csv=/path/to/prompts.csv \n```\nFor computing **ImageReward**, please refer to the official repository: https://github.com/zai-org/ImageReward\n\n\u003e [!NOTE]  \n\u003e Prompt CSV files used for evaluation are provided in the `datasets/` directory.\n\n\n\n## 🗂 Generate Custom Evaluation Dataset\nGenerate a dataset by searching for valid seeds and saving prompts + seeds into a CSV file.  \nPrompts are provided as `.txt` files (one per line). Example files are in `prompts/`.\nThe script also saves generated images and corresponding artifact masks.\nThe `seed` parameter specifies the starting seed from which the search begins\n\n```bash\npython src/generate_dataset.py \\\n  model=dev \\\n  seed=100000 \\\n  dataset.prompts_file=prompts/animals.txt \\\n  dataset.name=my_dataset \\\n  output.run_name=dataset_gen\n```\n\n\u003e [!NOTE]  \n\u003e Dataset generation is supported for **FLUX.1 [dev]**, **FLUX.1 [schnell]**, **FLUX.2 [dev]**, and **SDXL**.  \n\u003e To switch models, only the script name and the `model` value need to be changed:\n\u003e - `generate_dataset.py` → dev/schnell \n\u003e - `generate_dataset_flux2.py` → flux2dev\n\u003e - `generate_dataset_sdxl.py` → sdxl\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/gmum.github.io%2FDIAMOND%2F","html_url":"https://awesome.ecosyste.ms/projects/gmum.github.io%2FDIAMOND%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/gmum.github.io%2FDIAMOND%2F/lists"}