{"id":27426197,"url":"https://github.com/posgnu/guided-diffusion-tissues","last_synced_at":"2025-07-08T18:38:04.549Z","repository":{"id":74009024,"uuid":"524313425","full_name":"posgnu/guided-diffusion-tissues","owner":"posgnu","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-10T07:57:41.000Z","size":8605,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-14T12:53:04.239Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/posgnu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-08-13T05:46:18.000Z","updated_at":"2022-08-13T05:47:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"7acb3831-497e-4d6c-9f71-fc9d8143b88f","html_url":"https://github.com/posgnu/guided-diffusion-tissues","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/posgnu/guided-diffusion-tissues","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/posgnu%2Fguided-diffusion-tissues","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/posgnu%2Fguided-diffusion-tissues/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/posgnu%2Fguided-diffusion-tissues/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/posgnu%2Fguided-diffusion-tissues/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/posgnu","download_url":"https://codeload.github.com/posgnu/guided-diffusion-tissues/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/posgnu%2Fguided-diffusion-tissues/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264324674,"owners_count":23590934,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-14T12:31:36.374Z","updated_at":"2025-07-08T18:38:04.543Z","avatar_url":"https://github.com/posgnu.png","language":"Python","readme":"# guided-diffusion-tissues\n\nThis is the codebase for the project on improving the resolution of tissue images. The code for ESRGAN is in `Real-ESRGAN.zip`.\n\nWe aim to try various CNN models, including recently reliazed guided diffusion models. We heavily rely on the codebase for [Diffusion Models Beat GANS on Image Synthesis](http://arxiv.org/abs/2105.05233) [openai/guided-diffusion](https://github.com/openai/guided-diffusion).\n\n## Requirements\n* Python 3.9\n\n## Prerequisites\n```shellscript\npip install -e .\n```\n```shellscript\nexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/anaconda3/lib\n```\n\n## MPI\nFor running with `mpi`, you need to install it. I did `conda install -c conda-forge mpi4py`. \n\n# Data exploration\n\nCheck *notebooks*. \nWe decided to divide the dataset to the following groups: 70% training, 20% validation, 10% testing.\nValidation dataset will be used to visualize intermediate results, testing will be held out until the very end.\n\nValidation samples:\n```\n'Slide002-2.tif', 'Slide003-2.tif', 'Slide005-1.tif', 'Slide008-1.tif', 'Slide008-2.tif', 'Slide010-1.tif', 'Slide011-1.tif', 'Slide011-5.tif', 'Slide011-6.tif', 'Slide019-3.tif', 'Slide022-1.tif', 'Slide022-3.tif', 'Slide023-3.tif', 'Slide025-1.tif', 'Slide028-1.tif', 'Slide029-3.tif', 'Slide030-1.tif', 'Slide032-3.tif', 'Slide036-1.tif', 'Slide036-2.tif', 'Slide037-2.tif', 'Slide039-1.tif', 'Slide042-1.tif', 'Slide044-3.tif', 'Slide046-3.tif', 'Slide047-2.tif', 'Slide053-1.tif'\n```\n\nTesting samples:\n```\n'Slide008-3.tif', 'Slide011-4.tif', 'Slide013-2.tif', 'Slide014-2.tif', 'Slide019-1.tif', 'Slide019-2.tif', 'Slide022-4.tif', 'Slide031-1.tif', 'Slide034-3.tif', 'Slide035-1.tif', 'Slide044-2.tif', 'Slide045-1.tif', 'Slide045-2.tif', 'Slide045-3.tif', 'Slide052-2.tif'\n```\n\nThe IDs are already included as agruments in `scripts/super_res_training.py` \n\n# Training guided-diffusion model\n\nYou need to specify number of GPUs `-n`, location for storing models and all logger files `--logdir`. Other parameters were taken from the paper mentioned above.\nFor trying a model, I suggest to change the following arguments:\n`--log_interval 1`\n`--save_interval 1`\n`--tb_valid_im_num 1`\n\nAfter each `--save_interval`, the validation set is visualized to tensorboard. `--tb_valid_im_num` defines number of validation patches displayed in tensorboard. I wanted to add also assessment of the validation losses, but for some reason I have a CUDA error (will deal with this later).\n\nTraining from the scratch\n```sh \nNCCL_P2P_DISABLE=1 mpiexec -n 8 python3 scripts/super_res_train.py \\\n--model_path log/model240000.pt \\\n--patch_size 256 \\\n--data_dir /baldig/bioprojects2/BrownLab/Ptychography/Registered_Images2/high_res \\\n--log_dir log \\\n--diffusion_steps 1000 \\\n--noise_schedule \"linear\" \\\n--num_channels 192 \\\n--num_res_blocks 2 \\\n--num_head_channels 64 \\\n--attention_resolutions \"32,16,8\" \\\n--lr 1e-4 \\\n--log_interval 1000 --save_interval 10000 \\\n--batch_size 4 --tb_valid_im_num 8\n```\n\nPre-training\n```sh\nNCCL_P2P_DISABLE=1 mpiexec -n 8 python3 scripts/super_res_train.py \\\n--patch_size 256 \\\n--data_dir /home/kgw/guided-diffusion-tissues/ILSVRC/Data/CLS-LOC/test \\\n--val_data_dir /home/kgw/guided-diffusion-tissues/ILSVRC/Data/CLS-LOC/val \\\n--log_dir pre-train-log-256 \\\n--diffusion_steps 1000 \\\n--noise_schedule \"linear\" \\\n--num_channels 192 \\\n--num_res_blocks 2 \\\n--num_head_channels 64 \\\n--attention_resolutions \"32,16,8\" \\\n--lr 1e-4 \\\n--log_interval 100 --save_interval 10000 \\\n--batch_size 2 --tb_valid_im_num 8 \\\n--pre_train True\n```\n\nResume training\n```sh\nNCCL_P2P_DISABLE=1 mpiexec -n 8 python3 scripts/super_res_train.py \\\n--resume_checkpoint log-fine-tune/model640000.pt \\\n--patch_size 256 \\\n--data_dir /baldig/bioprojects2/BrownLab/Ptychography/Registered_Images2/high_res \\\n--log_dir log-fine-tune \\\n--diffusion_steps 1000 \\\n--noise_schedule \"linear\" \\\n--num_channels 192 \\\n--num_res_blocks 2 \\\n--num_head_channels 64 \\\n--attention_resolutions \"32,16,8\" \\\n--lr 1e-4 \\\n--log_interval 100 --save_interval 10000 \\\n--batch_size 14 --tb_valid_im_num 8\n```\n\nInference\n```sh\nNCCL_P2P_DISABLE=1 mpiexec -n 8 python3 scripts/super_res_sample.py \\\n--model_path log-fine-tune/model670000.pt \\\n--diffusion_steps 1000 \\\n--noise_schedule \"linear\" \\\n--num_channels 192 \\\n--num_res_blocks 2 \\\n--num_head_channels 64 \\\n--attention_resolutions \"32,16,8\" \\\n--base_samples /baldig/bioprojects2/BrownLab/Ptychography/Registered_Images2/low_res/Slide022-1.tif \\\n--log_dir test-result-256\n```\n\nSample patches\n```\nNCCL_P2P_DISABLE=1 mpiexec -n 8 python3 scripts/super_res_sample_patches.py \\\n--patch_size 256 \\\n--model_path log/model080000.pt \\\n--diffusion_steps 1000 \\\n--noise_schedule \"linear\" \\\n--num_channels 192 \\\n--num_res_blocks 2 \\\n--num_head_channels 64 \\\n--attention_resolutions \"32,16,8\" \\\n--base_samples /baldig/bioprojects2/BrownLab/Ptychography/Registered_Images2/low_res/Slide022-1.tif \\\n--log_dir test-result-256\n```\n\nUNet training\n```\npython3 unet_train.py /baldig/bioprojects2/BrownLab/Ptychography/Registered_Images2/high_res --gpus 1 --epochs 10000\n```\n\u003e Things to consider:\n\u003e\n\u003e Batch size is small because CUDA runs out of memory. Needs to be trained on larger nodes.\n\u003e\n\u003e Patch size is also small, however, if making it larger, the batch size should be even smaller.\n\u003e\n\u003e For now loss is just MSE. Maybe later we can add variational lower bound to the loss function.\n\n# Running tensorboard\nGo to the folder, which you indicated in `--log_dir`\n```sh\ntensorboard --logdir runs/ --port=6009 --bind_all\n```\nThe first raw of images correspond to low resolution input, the second row - high resolution target, the third row = prediction.\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fposgnu%2Fguided-diffusion-tissues","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fposgnu%2Fguided-diffusion-tissues","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fposgnu%2Fguided-diffusion-tissues/lists"}