{"id":13464394,"url":"https://github.com/victorca25/traiNNer","last_synced_at":"2025-03-25T11:31:35.207Z","repository":{"id":43253778,"uuid":"191514773","full_name":"victorca25/traiNNer","owner":"victorca25","description":"traiNNer: Deep learning framework for image and video super-resolution, restoration and image-to-image translation, for training and testing.","archived":false,"fork":false,"pushed_at":"2022-12-25T07:55:37.000Z","size":44191,"stargazers_count":290,"open_issues_count":16,"forks_count":39,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-10-29T17:48:59.580Z","etag":null,"topics":["bsrgan","cartoonization","convolutional-neural-networks","cyclegan","deblurring","denoising","esrgan","image-restoration","pix2pix","real-esrgan","real-sr","srflow","super-resolution","upscale"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/victorca25.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"patreon":"victorca25"}},"created_at":"2019-06-12T06:53:07.000Z","updated_at":"2024-10-28T03:24:05.000Z","dependencies_parsed_at":"2023-01-30T21:45:33.897Z","dependency_job_id":null,"html_url":"https://github.com/victorca25/traiNNer","commit_stats":null,"previous_names":["victorca25/basicsr"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorca25%2FtraiNNer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorca25%2FtraiNNer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorca25%2FtraiNNer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorca25%2FtraiNNer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/victorca25","download_url":"https://codeload.github.com/victorca25/traiNNer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245453979,"owners_count":20617956,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bsrgan","cartoonization","convolutional-neural-networks","cyclegan","deblurring","denoising","esrgan","image-restoration","pix2pix","real-esrgan","real-sr","srflow","super-resolution","upscale"],"created_at":"2024-07-31T14:00:42.127Z","updated_at":"2025-03-25T11:31:33.399Z","avatar_url":"https://github.com/victorca25.png","language":"Python","funding_links":["https://patreon.com/victorca25"],"categories":["Methods","Python"],"sub_categories":[],"readme":"# traiNNer\n\n[![Python Version](https://img.shields.io/badge/python-3-informational?style=flat)](https://python.org)\n[![License](https://img.shields.io/github/license/victorca25/traiNNer?style=flat)](https://github.com/victorca25/traiNNer/blob/master/LICENSE)\n[![DeepSource](https://deepsource.io/gh/victorca25/traiNNer.svg/?label=active+issues\u0026show_trend=true)](https://deepsource.io/gh/victorca25/traiNNer/?ref=repository-badge)\n[![Issues](https://img.shields.io/github/issues/victorca25/traiNNer?style=flat)](https://github.com/victorca25/traiNNer/issues)\n[![PR's Accepted](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat)](https://makeapullrequest.com)\n\ntraiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.\n\nHere you will find: boilerplate code for training and testing computer vision (CV) models, different methods and strategies integrated in a single pipeline and modularity to add and remove components as needed, including new network architectures and templates for different training strategies. The code is under a constant state of change, so if you find an issue or bug please open a [issue](https://github.com/victorca25/traiNNer/issues), a [discussion](https://github.com/victorca25/traiNNer/discussions) or write in one of the [Discord channels](##additional-help) for help.\n\nDifferent from other repositories, here the focus is not only on repeating previous papers' results, but to enable more people to train their own models more easily, using their own custom datasets, as well as integrating new ideas to increase the performance of the models. For these reasons, a lot of the code is made in order to automatically take care of fixing potential issues, whenever possible.\n\nDetails of the currently supported architectures can be found [here](https://github.com/victorca25/traiNNer/blob/master/docs/architectures.md).\n\nFor a changelog and general list of features of this repository, check [here](https://github.com/victorca25/traiNNer/blob/master/docs/changes.md).\n    \n## Table of Contents\n1.  [Dependencies](#dependencies)\n2.  [Codes](#codes)\n3.  [Usage](#usage)\n4.  [Pretrained models](#pretrained-models)\n5.  [Datasets](#datasets)\n6.  [How to help](#how-to-help)\n\n\n### Dependencies\n\n-   Python 3 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux))\n-   [PyTorch \u003e= 0.4.0](https://pytorch.org/). PyTorch \u003e= 1.7.0 required to enable certain features (SWA, AMP, others), as well as [torchvision](https://pytorch.org/vision/stable/index.html).\n-   NVIDIA GPU + [CUDA](https://developer.nvidia.com/cuda-downloads)\n-   Python packages: `pip install numpy opencv-python`\n-   `JSON` files can be used for the configuration option files, but in order to use `YAML`, the `PyYAML` python package is also a dependency: [`pip install PyYAML`](https://pyyaml.org/)\n\n#### Optional Dependencies\n\n-   Python package: [`pip install tensorboardX`](https://github.com/lanpa/tensorboardX), for visualizing curves.\n-   Python package: [`pip install lmdb`](https://github.com/jnwatson/py-lmdb), for lmdb database support.\n-   Python package: [`pip install scipy`](https://www.scipy.org/) to use [CEM](https://github.com/victorca25/traiNNer/blob/master/codes/models/modules/architectures/CEM/README.md).\n-   Python package: [`pip install Pillow`](https://python-pillow.org/) to use as an alternative image backend (default is OpenCV).\n-   Python package: [`pip install joblib`](https://joblib.readthedocs.io/) to train White-box Cartoonization (WBC) models.\n\n## Codes\n\nThis repository is a full framework for training different kinds of networks, with multiple enhancements and options. In [`./codes`](https://github.com/victorca25/traiNNer/tree/master/codes) you will find a more detailed explaination of the **code framework** ).\n\nYou will also find:\n1.  Some useful scripts. More details in [`./codes/scripts`](https://github.com/victorca25/traiNNer/tree/master/codes/scripts). \n2.  [Evaluation codes](https://github.com/victorca25/traiNNer/tree/master/metrics), e.g., PSNR/SSIM metric.\n\nAdditionally, it is complemented by other repositories like [DLIP](https://github.com/victorca25/DLIP), that can be used in order to extract estimated kernels and noise patches from real images, using a modified KernelGAN and patches extraction code. Detailed instructions about how to use the estimated kernels are available [here](https://github.com/victorca25/traiNNer/blob/master/docs/kernels.md)\n\n## Usage\n\n### Training\n\n#### Data and model preparation\n\nIn order to train your own models, you will need to create a [dataset](#datasets) consisting of images, and prepare these images, both considering [IO](https://github.com/victorca25/traiNNer/wiki/IO-speed) constrains, as well as the task the model should target. Detailed data preparation can be seen in [`codes/data`](https://github.com/victorca25/traiNNer/tree/master/codes/data).\n\n[**Pretrained models**](#pretrained-models) that can be used for fine-tuning are available.\n\nDetailed instructions on [how to train](https://github.com/victorca25/traiNNer/blob/master/docs/howtotrain.md) are also available.\n\nAugmentations strategies for training real-world models (blind SR) like [Real-SR](https://openaccess.thecvf.com/content_CVPRW_2020/papers/w31/Ji_Real-World_Super-Resolution_via_Kernel_Estimation_and_Noise_Injection_CVPRW_2020_paper.pdf), [BSRGAN](https://arxiv.org/pdf/2103.14006v1.pdf) and [Real-ESRGAN](https://arxiv.org/pdf/2107.10833.pdf) are provided via [presets](https://github.com/victorca25/traiNNer/tree/master/codes/options/presets) that define the blur, resizing and noise configurations, but many more [augmentations](https://github.com/victorca25/traiNNer/blob/master/docs/augmentations.md) are available to define custom training strategies.\n\n### How to Test\n\n#### For simple testing\nThe recommended way to get started with some of the models produced by the training codes available in this repository is by getting the [pretrained models](#pretrained-models) to be tested and run them in the companion repository [iNNfer](https://github.com/victorca25/iNNfer), with the purpose of model inference.\n\nAdditionally, you can also use a GUI (for [ESRGAN models](https://github.com/n00mkrad/cupscale), for [video](https://github.com/n00mkrad/flowframes)) or a smaller repo for inference (for [ESRGAN](https://github.com/JoeyBallentine/ESRGAN), for [video](https://github.com/JoeyBallentine/Video-Inference)). \n\nIf you are interested in obtaining results that can automatically return evaluation metrics, it is also possible to do inference of batches of images and some additional options with the instructions in [how to test](https://github.com/victorca25/traiNNer/blob/master/docs/howtotest.md).\n\n\n## Pretrained models\nThe most recent community pretrained models can be found in the [Wiki](https://upscale.wiki/wiki/Model_Database), Discord channels ([game upscale](https://discord.gg/nbB4A5F) and [animation upscale](https://discord.gg/vMaeuTEPh9)) and [nmkd's models](https://nmkd.de/?esrgan).\n\nFor more details about the original and experimental pretrained models, please see [`pretrained models`](https://github.com/victorca25/traiNNer/tree/master/docs/pretrained.md).\n\nYou can put the downloaded models in the default `experiments/pretrained_models` directory and use them in the options files with the corresponding network architectures.\n\n\n### Model interpolation\nModels that were trained using the same pretrained model or are derivates of the same pretrained model are able to be interpolated to combine the properties of both. The original author demostrated this by interpolating the PSNR pretrained model (which is not perceptually good, but results in smooth images) with the ESRGAN resulting models that have more details but sometimes is excessive to control a balance in the resulting images, instead of interpolating the resulting images from both models, giving much better results.\n\nThe capabilities of linearly interpolating models are also explored in \"DNI\": [Deep Network Interpolation for Continuous Imagery Effect Transition](https://xinntao.github.io/projects/DNI) (CVPR19) with very interesting results and examples. The script for interpolation can be found in the [net_interp.py](https://github.com/victorca25/traiNNer/blob/master/codes/scripts/net_interp.py) file. This is an alternative to create new models without additional training and also to create pretrained models for easier fine tuning. Below is an example of interpolating between a PSNR-oriented and a perceptual `ESRGAN` model (first row), and examples of interpolating `CycleGAN` style transfer models.\n\n\u003cp align=\"center\"\u003e\n   \u003cimg src=\"https://camo.githubusercontent.com/913baa366ba395595a9638ab6282a9cbb088ab98/68747470733a2f2f78696e6e74616f2e6769746875622e696f2f70726f6a656374732f444e495f7372632f7465617365722e6a7067\" height=\"300\"\u003e\n\u003c/p\u003e\n\nMore details and explanations of interpolation can be found [here](https://github.com/victorca25/traiNNer/wiki/Interpolation) in the Wiki.\n\n## Datasets\n\nMany [datasets](https://github.com/victorca25/traiNNer/blob/master/docs/datasets.md) are publicly available and used to train models in a way that can be benchmarked and compared with other models. You are also able to create your own datasets with your own images.\n\nAny dataset can be augmented to expose the model to information that might not be available in the images, such a noise and blur. For this reason, a [data augmentation](https://github.com/victorca25/traiNNer/wiki/Dataset-Augmentation) pipeline has been added to the options in this repository. It is also possible to add other types of augmentations, such as `Batch Augmentations` to apply them to minibatches instead of single images. Lastly, if your dataset is small, you can make use of `Differential Augmentations` to allow the discriminator to extract more information from the available images and train better models. More information can be found in the [augmentations](https://github.com/victorca25/traiNNer/blob/master/docs/augmentations.md) document.\n\n# How to help\n\nThere are multiple ways to help this project. The first one is by using it and trying to train your own models. You can open an [issue](https://github.com/victorca25/traiNNer/issues) if you find any bugs or start a [discussion](https://github.com/victorca25/traiNNer/discussions) if you have ideas, questions or would like to showcase your results.\n\nIf you would like to contribute in the form of adding or fixing code, you can do so by cloning this repo and creating a [PR](https://github.com/victorca25/traiNNer/pulls). Ideally, it's better for PR to be precise and not changing many parts of the code at the same time, so it can be reviewed and tested. If possible, open an issue or discussion prior to creating the PR and we can talk about any ideas.\n\nYou can also join the [discord servers](#additional-Help) and share results and questions with other users.\n\nLastly, after it has been suggested many times before, now there are options to donate to show your support to the project and help stir it in directions that will make it even more useful. Below you will find those options that were suggested.\n\n\u003cp align=\"left\"\u003e\n   \u003ca href=\"https://patreon.com/victorca25\"\u003e\n      \u003cimg src=\"https://github.githubassets.com/images/modules/site/icons/funding_platforms/patreon.svg\" height=\"30\"\u003e\n      Patreon\n   \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"left\"\u003e\n   \u003ca href=\"https://user-images.githubusercontent.com/41912303/121814560-fba1fc80-cc71-11eb-9b98-17c3ce0f06d6.png\"\u003e\n      \u003cimg src=\"https://user-images.githubusercontent.com/41912303/121814516-ca293100-cc71-11eb-9ddf-ffda840cd36d.png\" height=\"30\"\u003e\n      \u003cimg src=\"https://user-images.githubusercontent.com/41912303/121814560-fba1fc80-cc71-11eb-9b98-17c3ce0f06d6.png\" height=\"30\"\u003e\n   \u003c/a\u003e\n   Bitcoin Address: 1JyWsAu7aVz5ZeQHsWCBmRuScjNhCEJuVL\n\u003c/p\u003e\n\n\u003cp align=\"left\"\u003e\n   \u003ca href=\"https://user-images.githubusercontent.com/41912303/121814692-aa463d00-cc72-11eb-99b2-c1bae3f63fdc.png\"\u003e\n      \u003cimg src=\"https://user-images.githubusercontent.com/41912303/121814599-36a43000-cc72-11eb-974a-146661e5e665.png\" height=\"30\"\u003e\n      \u003cimg src=\"https://user-images.githubusercontent.com/41912303/121814692-aa463d00-cc72-11eb-99b2-c1bae3f63fdc.png\" height=\"30\"\u003e\n   \u003c/a\u003e\n   Ethereum Address: 0xa26AAb3367D34457401Af3A5A0304d6CbE6529A2\n\u003c/p\u003e\n\n* * *\n## Additional Help\n\nIf you have any questions, we have a couple of discord servers ([game upscale](https://discord.gg/nbB4A5F) and [animation upscale](https://discord.gg/vMaeuTEPh9)) where you can ask them and a [Wiki](https://upscale.wiki/) with more information.\n\n* * *\n\n## Acknowledgement\n\nCode architecture is originally inspired by [pytorch-cyclegan](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) and the first version of [BasicSR](https://github.com/xinntao/BasicSR).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorca25%2FtraiNNer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvictorca25%2FtraiNNer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorca25%2FtraiNNer/lists"}