{"id":13592764,"url":"https://github.com/Warvito/generative_brain_controlnet","last_synced_at":"2025-04-09T02:31:14.330Z","repository":{"id":173335152,"uuid":"650534498","full_name":"Warvito/generative_brain_controlnet","owner":"Warvito","description":"Repository to train ControlNet on Brain data (UK BIOBANK) using MONAI Generative Models","archived":false,"fork":false,"pushed_at":"2023-07-02T08:07:27.000Z","size":427,"stargazers_count":74,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-02T02:02:17.432Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Warvito.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-07T09:17:43.000Z","updated_at":"2024-10-21T22:35:40.000Z","dependencies_parsed_at":"2024-01-16T22:20:07.709Z","dependency_job_id":"5957f5be-e537-4bba-b4bc-9aa77e05c2a6","html_url":"https://github.com/Warvito/generative_brain_controlnet","commit_stats":null,"previous_names":["warvito/generative_brain_controlnet"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Warvito%2Fgenerative_brain_controlnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Warvito%2Fgenerative_brain_controlnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Warvito%2Fgenerative_brain_controlnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Warvito%2Fgenerative_brain_controlnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Warvito","download_url":"https://codeload.github.com/Warvito/generative_brain_controlnet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223356296,"owners_count":17132118,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T16:01:12.983Z","updated_at":"2024-11-06T14:30:18.085Z","avatar_url":"https://github.com/Warvito.png","language":"Python","funding_links":[],"categories":["Imaging phenotypes"],"sub_categories":["Brain MRI"],"readme":"# ControlNet for Brain T1w Images Generation from FLAIR images using MONAI Generative Models\n\nScript to train a ControlNet (from [Adding Conditional Control to Text-to-Image Diffusion Models](https://arxiv.org/abs/2302.05543))\non UK BIOBANK dataset to transform FLAIRs to T1w 2D images using [MONAI Generative Models](https://github.com/Project-MONAI/GenerativeModels)\npackage.\n\n![ControlNet Samples](https://github.com/Warvito/generative_brain_controlnet/blob/main/assets/figure_samples.png?raw=true)\n\nThis repository is part of this Medium post.\n\n## Instructions\n### Preprocessing\nAfter Downloading UK Biobank dataset and preprocessing it, we obtain the list of the image paths for the T1w and FLAIR\nimages. For that, we use the following script:\n\n1) `src/python/preprocessing/create_png_dataset.py` -  Create png images from the nifti files\n2) `src/python/preprocessing/create_ids.py` -  Create files with datalist for training, validation and test\n\n### Training\nAfter we obtain the paths, we can train the models using similar commands as in the following files (note: This project was\nexecuted on a cluster with RunAI platform):\n\n1) `cluster/runai/training/stage1.sh` - Command to start to execute in the server the training the first stage of the model.\nThe main python script in for this is the `src/python/training/train_aekl.py` script. The `--volume` flags indicate how the dataset\nis mounted in the Docker container.\n3) `cluster/runai/training/ldm.sh` - Command to start to execute in the server the training the diffusion model on the latent representation.\nThe main python script in for this is the `src/python/training/train_ldm.py` script. The `--volume` flags indicate how the dataset\nis mounted in the Docker container.\n4) `cluster/runai/training/controlnet.sh` - Command to start to execute in the server the training the ControlNet model using the pretrained LDM.\nThe main python script in for this is the `src/python/training/train_controlnet.py` script. The `--volume` flags indicate how the dataset\nis mounted in the Docker container.\n\nThese `.sh` files indicates which parameters and configuration file was used for training, as well how the host directories\nwere mounted in the used Docker container.\n\n\n### Inference and evaluation\nFinally, we converted the mlflow model to .pth files (for easily loading with MONAI), sampled images from the diffusion\nmodel and controlnet, and evaluated the models. The following is the list of execution for inference and evaluation:\n\n1) `src/python/testing/convert_mlflow_to_pytorch.py` - Convert mlflow model to .pth files\n2) `src/python/testing/sample_t1w.py` - Sample T1w images from the diffusion model without using contditioning.\n3) `cluster/runai/testing/sample_flair_to_t1w.py` - Sample T1w images from the controlnet using the test set's FLAIR\nimages as conditionings.\n4) `src/python/testing/compute_msssim_reconstruction.py` - Measure the mean structural similarity index between images and\nreconstruction to measure the preformance of the autoencoder.\n5) `src/python/testing/compute_msssim_sample.py` - Measure the mean structural similarity index between samples in order\nto measure the diversity of the synthetic data.\n6) `src/python/testing/compute_fid.py` - Compute FID score between generated images and real images.\n7) `src/python/testing/compute_controlnet_performance.py` - Compute the performance of the controlnet using MAE, PSNR and\nMS-SSIM metrics.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWarvito%2Fgenerative_brain_controlnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FWarvito%2Fgenerative_brain_controlnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWarvito%2Fgenerative_brain_controlnet/lists"}