{"id":13543538,"url":"https://github.com/yu4u/noise2noise","last_synced_at":"2025-05-16T19:03:12.733Z","repository":{"id":41168833,"uuid":"141627143","full_name":"yu4u/noise2noise","owner":"yu4u","description":"An unofficial and partial Keras implementation of \"Noise2Noise: Learning Image Restoration without Clean Data\"","archived":false,"fork":false,"pushed_at":"2021-08-12T12:14:29.000Z","size":9589,"stargazers_count":1087,"open_issues_count":37,"forks_count":236,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-04-03T19:14:05.070Z","etag":null,"topics":["convolutional-neural-networks","deep-learning","denoising","keras","noise2noise"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yu4u.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-07-19T20:22:11.000Z","updated_at":"2025-03-11T10:25:10.000Z","dependencies_parsed_at":"2022-07-16T23:31:01.711Z","dependency_job_id":null,"html_url":"https://github.com/yu4u/noise2noise","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu4u%2Fnoise2noise","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu4u%2Fnoise2noise/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu4u%2Fnoise2noise/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yu4u%2Fnoise2noise/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yu4u","download_url":"https://codeload.github.com/yu4u/noise2noise/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248607079,"owners_count":21132470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","deep-learning","denoising","keras","noise2noise"],"created_at":"2024-08-01T11:00:32.710Z","updated_at":"2025-04-12T17:39:52.514Z","avatar_url":"https://github.com/yu4u.png","language":"Python","funding_links":[],"categories":["Denoising Algorithms"],"sub_categories":[],"readme":"# Noise2Noise\n\nThis is an unofficial and partial Keras implementation of \"Noise2Noise: Learning Image Restoration without Clean Data\" [1].\n\nThere are several things different from the original paper\n(but not a fatal problem to see how the noise2noise training framework works):\n- Training dataset (orignal: ImageNet, this repository: [2])\n- Model (original: RED30 [3], this repository: SRResNet [4] or UNet [5])\n\nUpdates:\n- [Sep. 21, 2018] Random-valued impulse noise model and L0 loss were added\n- [Aug. 25, 2018] UNet model can be used in training\n- [Aug. 25, 2018] Add trained weights\n\n## Dependencies\n- Keras \u003e= 2.1.2, TensorFlow, NumPy, OpenCV\n\n## Train Noise2Noise\n\n### Download Dataset\n\n```bash\nmkdir dataset\ncd dataset\nwget https://cv.snu.ac.kr/research/VDSR/train_data.zip\nwget https://cv.snu.ac.kr/research/VDSR/test_data.zip\nunzip train_data.zip\nunzip test_data.zip\ncd ..\n```\n\nAny dataset can be used in training and validation instead of the above dataset.\n\n### Train Model\nPlease see `python3 train.py -h` for optional arguments.\n\n\n#### Train with Gaussian noise\n```bash\n# train model using (noise, noise) pairs (noise2noise)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --output_path gaussian\n\n# train model using (noise, clean) paris (standard training)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --target_noise_model clean --output_path clean\n```\n\n\n#### Train with text insertion\n\n```bash\n# train model using (noise, noise) pairs (noise2noise)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --source_noise_model text,0,50 --target_noise_model text,0,50 --val_noise_model text,25,25 --loss mae --output_path text_noise\n\n# train model using (noise, clean) paris (standard training)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --source_noise_model text,0,50 --target_noise_model clean --val_noise_model text,25,25 --loss mae --output_path text_clean\n```\n\n#### Train with random-valued impulse noise\n\n```bash\n# train model using (noise, noise) pairs (noise2noise)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --source_noise_model impulse,0,95 --target_noise_model impulse,0,95 --val_noise_model impulse,70,70 --loss l0 --output_path impulse_noise\n\n# train model using (noise, clean) paris (standard training)\npython3 train.py --image_dir dataset/291 --test_dir dataset/Set14 --image_size 128 --batch_size 8 --lr 0.001 --source_noise_model impulse,0,95 --target_noise_model clean --val_noise_model impulse,70,70 --loss l0 --output_path impulse_clean\n```\n\n##### Model architectures\nWith `--model unet`, UNet model can be trained instead of SRResNet.\n\n##### Resume training\nWith `--weight path/to/weight/file`, training can be resumed with trained weights.\n\n\n### Noise Models\nUsing `source_noise_model`, `target_noise_model`, and `val_noise_model` arguments,\narbitrary noise models can be set for source images, target images, and validatoin images respectively.\nDefault values are taken from the experiment in [1].\n\n- Gaussian noise\n  - gaussian,min_stddev,max_stddev (e.g. gaussian,0,50)\n- Clean target\n  - clean\n- Text insertion\n  - text,min_occupancy,max_occupancy (e.g. text,0,50)\n- Random-valued impulse noise\n  - impulse,min_occupancy,max_occupancy (e.g. impulse,0,50)\n\nYou can see how these noise models work by:\n\n```bash\npython3 noise_model.py --noise_model text,0,95\n```\n\n### Results\n#### Plot training history\n\n```bash\npython3 plot_history.py --input1 gaussian --input2 clean\n```\n\n##### Gaussian noise\n\u003cimg src=\"result/val_loss.png\" width=\"480px\"\u003e\n\n\n\u003cimg src=\"result/val_PSNR.png\" width=\"480px\"\u003e\n\nFrom the above result, I confirm that we can train denoising model using noisy targets\nbut it is not comparable to the model trained using clean targets.\nIf UNet is used, the result becomes 29.67 (noisy targets) vs. 30.14 (clean targets).\n\n##### Text insertion\n\u003cimg src=\"result/val_loss_text.png\" width=\"480px\"\u003e\n\n\n\u003cimg src=\"result/val_PSNR_text.png\" width=\"480px\"\u003e\n\n#### Random-valued impulse noise\n\u003cimg src=\"result/val_loss_impulse.png\" width=\"480px\"\u003e\n\n\n\u003cimg src=\"result/val_PSNR_impulse.png\" width=\"480px\"\u003e\n\n\n#### Check denoising result\n\n```bash\npython3 test_model.py --weight_file [trained_model_path] --image_dir dataset/Set14\n```\n\nThe detailed options are:\n\n```bash\noptional arguments:\n  -h, --help            show this help message and exit\n  --image_dir IMAGE_DIR\n                        test image dir (default: None)\n  --model MODEL         model architecture ('srresnet' or 'unet') (default:\n                        srresnet)\n  --weight_file WEIGHT_FILE\n                        trained weight file (default: None)\n  --test_noise_model TEST_NOISE_MODEL\n                        noise model for test images (default: gaussian,25,25)\n  --output_dir OUTPUT_DIR\n                        if set, save resulting images otherwise show result\n                        using imshow (default: None)\n```\nThis script adds noise using `test_noise_model` to each image in `image_dir` and performs denoising.\nIf you want to perform denoising to already noisy images, use `--test_noise_model clean`.\n\n##### Gaussian noise\nDenoising result by clean target model (left to right: original, degraded image, denoised image):\n\n\u003cimg src=\"result/baby_GT_clean.png\" width=\"800px\"\u003e\n\nDenoising result by noise target model:\n\n\u003cimg src=\"result/baby_GT_gaussian.png\" width=\"800px\"\u003e\n\n##### Text insertion\nDenoising result by clean target model\n\n\u003cimg src=\"result/baby_GT_text_clean.png\" width=\"800px\"\u003e\n\nDenoising result by noise target model:\n\n\u003cimg src=\"result/baby_GT_text_noise.png\" width=\"800px\"\u003e\n\n##### Random-valued impulse noise\nDenoising result by clean target model\n\n\u003cimg src=\"result/baby_GT_impulse_clean.png\" width=\"800px\"\u003e\n\nDenoising result by noise target model:\n\n\u003cimg src=\"result/baby_GT_impulse_noise.png\" width=\"800px\"\u003e\n\n\n#### Trained weights\n\n[Gaussian noise, noise target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.040-87.447-29.13496_gauss_noise.hdf5)\n\n[Gaussian noise, clean target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.056-66.803-30.57923_gauss_clean.hdf5)\n\n[Text insertion, noise target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.057-4.796-27.68533_text_noise.hdf5)\n\n[Text insertion, clean target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.056-4.172-28.07752_text_clean.hdf5)\n\n[Random-valued impulse noise, noise target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.047-2.317-24.30238_impulse_noise.hdf5)\n\n[Random-valued impulse noise, clean target](https://github.com/yu4u/noise2noise/releases/download/v0.1/weights.038-4.547-24.81654_impulse_clean.hdf5)\n\n### TODOs\n\n- [x] Compare (noise, clean) training and (noise, noise) training\n- [x] Add different noise models\n- [x] Write readme\n\n## References\n\n[1] J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, \nT. Aila, \"Noise2Noise: Learning Image Restoration without Clean Data,\" in Proc. of ICML, 2018.\n\n[2] J. Kim, J. K. Lee, and K. M. Lee, \"Accurate Image Super-Resolution Using Very Deep Convolutional Networks,\" in Proc. of CVPR, 2016.\n\n[3] X.-J. Mao, C. Shen, and Y.-B. Yang, \"Image\nRestoration Using Convolutional Auto-Encoders with\nSymmetric Skip Connections,\" in Proc. of NIPS, 2016.\n\n[4] C. Ledig, et al., \"Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,\" in Proc. of CVPR, 2017.\n\n[5] O. Ronneberger, P. Fischer, and T. Brox, \"U-Net: Convolutional Networks for Biomedical Image Segmentation,\" in MICCAI, 2015.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyu4u%2Fnoise2noise","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyu4u%2Fnoise2noise","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyu4u%2Fnoise2noise/lists"}