{"id":13737903,"url":"https://github.com/jolibrain/joliGEN","last_synced_at":"2025-05-08T15:32:05.965Z","repository":{"id":37095936,"uuid":"288411806","full_name":"jolibrain/joliGEN","owner":"jolibrain","description":"Generative AI Image and Video Toolset with GANs and Diffusion for Real-World Applications","archived":false,"fork":false,"pushed_at":"2025-02-12T16:39:53.000Z","size":14128,"stargazers_count":258,"open_issues_count":22,"forks_count":37,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-12T12:54:33.909Z","etag":null,"topics":["augmented-reality","deep-learning","diffusion-models","gan","generative-model","image-generation","image-to-image","pytorch"],"latest_commit_sha":null,"homepage":"https://www.joligen.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jolibrain.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-08-18T09:26:25.000Z","updated_at":"2025-04-08T08:05:08.000Z","dependencies_parsed_at":"2023-10-13T02:57:04.576Z","dependency_job_id":"13c1c9b6-35b5-42bb-9e0f-6b09358dd97e","html_url":"https://github.com/jolibrain/joliGEN","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jolibrain%2FjoliGEN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jolibrain%2FjoliGEN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jolibrain%2FjoliGEN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jolibrain%2FjoliGEN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jolibrain","download_url":"https://codeload.github.com/jolibrain/joliGEN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253096265,"owners_count":21853567,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["augmented-reality","deep-learning","diffusion-models","gan","generative-model","image-generation","image-to-image","pytorch"],"created_at":"2024-08-03T03:02:05.301Z","updated_at":"2025-05-08T15:32:05.959Z","avatar_url":"https://github.com/jolibrain.png","language":"Python","funding_links":[],"categories":["Python","Open Source Frameworks","Benchmarks"],"sub_categories":["General-Purpose Machine Learning","Others"],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"imgs/joligen.svg\" width=\"512\"\u003e\n\u003c/div\u003e\n\n[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Docs](https://github.com/jolibrain/joliGEN/actions/workflows/github-actions-doc-options-update.yml/badge.svg)](https://github.com/jolibrain/joliGEN/actions/workflows/github-actions-doc-options-update.yml)\n[![Website](https://img.shields.io/website-up-down-green-red/http/joligen.com/doc.svg)](https://www.joligen.com/doc)\n\n\u003ch1 align=\"center\"\u003eGenerative AI Image \u0026 Video Toolset with GANs, Diffusion and Consistency Models for Real-World Applications\u003c/h1\u003e\n\n**JoliGEN** is an integrated framework for training custom generative AI image-to-image models\n\nMain Features:\n\n- JoliGEN implements both **GAN, Diffusion and Consistency models** for unpaired and paired image to image translation tasks, including domain and style adaptation with conservation of semantics such as image and object classes, masks, ...\n\n- JoliGEN generative AI capabilities are targeted at real world applications such as **Controled Image Generation**, **Augmented Reality**, **Dataset Smart Augmentation** and object insertion, **Synthetic to Real** transforms.\n\n- JoliGEN allows for fast and stable training with astonishing results. A [**server with REST API**](https://www.joligen.com/doc/server.html#running-joligen-server) is provided that allows for simplified deployment and usage.\n\n- JoliGEN has a large scope of [options and parameters](https://www.joligen.com/doc/options.html). To not get overwhelmed, follow the simple [Quickstarts](https://www.joligen.com/doc/quickstart_ddpm.html). There are then links to more detailed documentation on models, dataset formats, and data augmentation.\n\n### Useful links\n\n- [JoliGEN documentation](https://www.joligen.com/doc/)\n- [GAN Quickstart](https://www.joligen.com/doc/quickstart_gan.html)\n- [Diffusion Quickstart](https://www.joligen.com/doc/quickstart_ddpm.html)\n- [Datasets](https://www.joligen.com/doc/datasets.html)\n- [Training Tips](https://www.joligen.com/doc/tips.html)\n\n## Use cases\n\n- **AR and metaverse**: replace any image element with super-realistic objects\n- **Image manipulation**: seamlessly insert or remove objects/elements in images\n- **Image to image translation while preserving semantics**, e.g. existing source dataset annotations\n- **Simulation to reality** translation while preserving elements, metrics, ...\n- **Image generation to enrich datasets**, e.g. counter dataset imbalance, increase test sets, ...\n\nThis is achieved by combining powerful and customized generator architectures, bags of discriminators, and configurable neural networks and losses that ensure conservation of fundamental elements between source and target images.\n\n## Example results\n\n### Satellite imagery inpainting\n\nFill up missing areas with diffusion network\n\n![xview_inpainting_res1](https://github.com/jolibrain/joliGEN/assets/3530657/9dc71c03-eeea-4811-ab1d-edfd3b078b19)\n\n\n### Image translation while preserving the class\n\nMario to Sonic while preserving the action (running, jumping, ...)\n\n![Clipboard - June 6, 2022 9 44 PM](https://user-images.githubusercontent.com/3530657/196461791-9ff55a47-1e74-4ee7-ad3b-0a915dee6ae6.png)\n![Clipboard - June 5, 2022 12 02 PM](https://user-images.githubusercontent.com/3530657/196461802-21d3015b-f5e8-467b-9096-78fcabd1f57b.png)\n\n### Object insertion\n\nVirtual Try-On with Diffusion\n\n![273150788-32990dc1-ebd7-401a-be51-85adeef3b508](https://github.com/jolibrain/joliGEN/assets/3530657/e3fc87a9-66be-4921-8abe-7dae479a2b10)\n\n\nCar insertion (BDD100K) with Diffusion\n![image](https://user-images.githubusercontent.com/3530657/196428508-3eae3415-8e15-4505-9e97-41c0ba99350e.png)\n![image](https://user-images.githubusercontent.com/3530657/196428593-6ad8e229-368a-4714-a1cc-8aa8210beaad.png)\n\nGlasses insertion (FFHQ) with Diffusion\n\n\u003cimg src=\"https://github.com/jolibrain/joliGEN/assets/3530657/eba7920d-4430-4f46-b65c-6cf2267457b0\" alt=\"drawing\" width=\"512\"/\u003e\n\u003cimg src=\"https://github.com/jolibrain/joliGEN/assets/3530657/ef908a7f-375f-4d0a-afec-72d1ee7eaafe\" alt=\"drawing\" width=\"512\"/\u003e\n\n### Object removal\n\nGlasses removal with GANs\n\n![Clipboard - November 9, 2022 4_33 PM](https://user-images.githubusercontent.com/3530657/200873590-6d1abe9a-7d86-458a-a9a5-97a1bcf4b816.png)\n![Clipboard - November 9, 2022 10_40 AM](https://user-images.githubusercontent.com/3530657/200873601-e8c2d165-af58-4b39-a0bf-ecab510981c5.png)\n\n### Style transfer while preserving label boxes (e.g. cars, pedestrians, street signs, ...)\n\nDay to night (BDD100K) with Transformers and GANs\n![image](https://user-images.githubusercontent.com/3530657/196472056-b342c326-056f-4680-ad8d-4bf932b1404a.png)\n\nClear to snow (BDD100K) by applying a generator multiple times to add snow incrementally\n![image](https://user-images.githubusercontent.com/3530657/196426503-bfaee698-b135-493f-81a6-644881cc1a5c.png)\n\nClear to overcast (BDD100K)\n![image](https://user-images.githubusercontent.com/3530657/196426571-f7e6189b-fa3a-4f5e-b6d5-fd580d14f29c.png)\n\nClear to rainy (BDD100K)\n![image](https://user-images.githubusercontent.com/3530657/196426461-e983c48f-ce19-4e83-a490-7a73b28c8181.png)\n![image](https://user-images.githubusercontent.com/3530657/196426623-deb7c00d-77e7-448e-827f-2423fd76b0ef.png)\n\n## Features\n\n- SoTA image to image translation\n- Semantic consistency: conservation of labels of many types: bounding boxes, masks, classes.\n- SoTA discriminator models: [projected](https://arxiv.org/abs/2111.01007), [vision_aided](https://arxiv.org/abs/2112.09130), custom transformers.\n- Advanced generators: [real-time](https://github.com/jolibrain/joliGEN/blob/chore_new_readme/models/modules/resnet_architecture/resnet_generator.py#L388), [transformers](https://arxiv.org/abs/2203.16015), [hybrid transformers-CNN](https://github.com/jolibrain/joliGEN/blob/chore_new_readme/models/modules/segformer/segformer_generator.py#L95), [Attention-based](https://arxiv.org/abs/1911.11897), [UNet with attention](https://github.com/jolibrain/joliGEN/blob/chore_new_readme/models/modules/unet_generator_attn/unet_generator_attn.py#L323), [HDiT](https://arxiv.org/abs/2401.11605)\n- Multiple models based on adversarial and diffusion generation: [CycleGAN](https://arxiv.org/abs/1703.10593), [CyCADA](https://arxiv.org/abs/1711.03213), [CUT](https://arxiv.org/abs/2007.15651), [Palette](https://arxiv.org/abs/2111.05826)\n- GAN data augmentation mechanisms: [APA](https://arxiv.org/abs/2111.06849), discriminator noise injection, standard image augmentation, online augmentation through sampling around bounding boxes\n- Output quality metrics: FID, PSNR, KID, ...\n- Server with [REST API](https://www.joligen.com/doc/API.html)\n- Support for both CPU and GPU\n- [Dockerized server](https://www.joligen.com/doc/docker.html)\n- Production-grade deployment in C++ via [DeepDetect](https://github.com/jolibrain/deepdetect/)\n\n---\n\n## Code format and Contribution\n\nIf you want to contribute please use [black](https://github.com/psf/black) code format.\nInstall:\n\n```\npip install black \n```\n\nUsage :\n\n```\nblack .\n```\n\nIf you want to format the code automatically before every commit :\n\n```\npip install pre-commit\npre-commit install\n```\n\n## Authors\n\n**JoliGEN** is created and developed by [Jolibrain](https://www.jolibrain.com/).\n\nCode structure is inspired by [pytorch-CycleGAN-and-pix2pix](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix), [CUT](https://github.com/taesungp/contrastive-unpaired-translation), [AttentionGAN](https://github.com/Ha0Tang/AttentionGAN), [MoNCE](https://github.com/fnzhan/MoNCE), [Palette](https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models) among others.\n\nElements from JoliGEN are supported by the French National AI program [\"Confiance.AI\"](https://www.confiance.ai/en/)\n\nContact: \u003ccontact@jolibrain.com\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjolibrain%2FjoliGEN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjolibrain%2FjoliGEN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjolibrain%2FjoliGEN/lists"}