{"id":13561951,"url":"https://github.com/ai-forever/ru-dalle","last_synced_at":"2025-05-15T18:04:37.869Z","repository":{"id":37544951,"uuid":"422998569","full_name":"ai-forever/ru-dalle","owner":"ai-forever","description":"Generate images from texts. In Russian","archived":false,"fork":false,"pushed_at":"2023-01-10T14:07:02.000Z","size":28230,"stargazers_count":1647,"open_issues_count":37,"forks_count":245,"subscribers_count":37,"default_branch":"master","last_synced_at":"2025-03-31T22:16:57.331Z","etag":null,"topics":["dalle","image-generation","openai","python","pytorch","russian","russian-language","text-to-image","transformer"],"latest_commit_sha":null,"homepage":"https://rudalle.ru/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ai-forever.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-30T21:55:52.000Z","updated_at":"2025-02-25T18:28:36.000Z","dependencies_parsed_at":"2023-02-08T19:01:03.887Z","dependency_job_id":null,"html_url":"https://github.com/ai-forever/ru-dalle","commit_stats":null,"previous_names":["sberbank-ai/ru-dalle"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fru-dalle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fru-dalle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fru-dalle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ai-forever%2Fru-dalle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ai-forever","download_url":"https://codeload.github.com/ai-forever/ru-dalle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247744333,"owners_count":20988783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dalle","image-generation","openai","python","pytorch","russian","russian-language","text-to-image","transformer"],"created_at":"2024-08-01T13:01:02.953Z","updated_at":"2025-04-07T23:06:57.020Z","avatar_url":"https://github.com/ai-forever.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# ruDALL-E\n### Generate images from texts\n\n[![Apache license](https://img.shields.io/badge/License-Apache-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)\n[![Downloads](https://pepy.tech/badge/rudalle)](https://pepy.tech/project/rudalle)\n[![Coverage Status](https://codecov.io/gh/sberbank-ai/ru-dalle/branch/master/graphs/badge.svg)](https://codecov.io/gh/sberbank-ai/ru-dalle)\n[![pipeline](https://gitlab.com/shonenkov/ru-dalle/badges/master/pipeline.svg)](https://gitlab.com/shonenkov/ru-dalle/-/pipelines)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/sberbank-ai/ru-dalle/master.svg)](https://results.pre-commit.ci/latest/github/sberbank-ai/ru-dalle/master)\n\n```\npip install rudalle==1.1.3\n```\n### 🤗 HF Models:\n[ruDALL-E Malevich (XL)](https://huggingface.co/sberbank-ai/rudalle-Malevich) \\\n[ruDALL-E Emojich (XL)](https://huggingface.co/sberbank-ai/rudalle-Emojich) (readme [here](https://github.com/sberbank-ai/ru-dalle/blob/master/Emojich.md)) \\\n[ruDALL-E Surrealist (XL)](https://huggingface.co/shonenkov-AI/rudalle-xl-surrealist) \\\nruDALL-E Kandinsky (XXL) (soon)\n\n### Minimal Example:\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1RztuaPetwz-QGHHpwlE5WAp3J3Ss70_4?usp=sharing)\n[![Kaggle](https://kaggle.com/static/images/open-in-kaggle.svg)](https://www.kaggle.com/shonenkov/rudalle-example-generation)\n[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/anton-l/rudall-e)\n\n**Example usage ruDALL-E Malevich (XL) with 3.5GB vRAM!**\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1AoolDYePUpPkRCKIu0cP9zV7lX5QGD3Z?usp=sharing)\n\n**Finetuning example**\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Tb7J4PvvegWOybPfUubl5O7m5I24CBg5?usp=sharing)\n\n### generation by ruDALLE:\n```python\nimport ruclip\nfrom rudalle.pipelines import generate_images, show, super_resolution, cherry_pick_by_ruclip\nfrom rudalle import get_rudalle_model, get_tokenizer, get_vae, get_realesrgan\nfrom rudalle.utils import seed_everything\n\n# prepare models:\ndevice = 'cuda'\ndalle = get_rudalle_model('Malevich', pretrained=True, fp16=True, device=device)\ntokenizer = get_tokenizer()\nvae = get_vae(dwt=True).to(device)\n\n# pipeline utils:\nrealesrgan = get_realesrgan('x2', device=device)\nclip, processor = ruclip.load('ruclip-vit-base-patch32-384', device=device)\nclip_predictor = ruclip.Predictor(clip, processor, device, bs=8)\ntext = 'радуга на фоне ночного города'\n\nseed_everything(42)\npil_images = []\nscores = []\nfor top_k, top_p, images_num in [\n    (2048, 0.995, 24),\n]:\n    _pil_images, _scores = generate_images(text, tokenizer, dalle, vae, top_k=top_k, images_num=images_num, bs=8, top_p=top_p)\n    pil_images += _pil_images\n    scores += _scores\n\nshow(pil_images, 6)\n```\n![](pics/malevich/rainbow-full.png)\n### auto cherry-pick by ruCLIP:\n```python\ntop_images, clip_scores = cherry_pick_by_ruclip(pil_images, text, clip_predictor, count=6)\nshow(top_images, 3)\n```\n![](pics/malevich/rainbow-cherry-pick.png)\n### super resolution:\n```python\nsr_images = super_resolution(top_images, realesrgan)\nshow(sr_images, 3)\n```\n![](pics/malevich/rainbow-super-resolution.png)\n\n```python\ntext, seed = 'красивая тян из аниме', 6955\n```\n![](pics/malevich/anime-girl-super-resolution.png)\n\n\n### Image Prompt\nsee `jupyters/ruDALLE-image-prompts-A100.ipynb`\n```python\ntext, seed = 'Храм Василия Блаженного', 42\nskyes = [red_sky, sunny_sky, cloudy_sky, night_sky]\n```\n![](pics/malevich/russian-temple-image-prompt.png)\n\n\n### VideoDALL-E | ru[CogVideo](https://github.com/THUDM/CogVideo) by [@cene555](https://github.com/cene555)\n**Video generation example**\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1A_3Oe9r9DP3Ayd6DPvqKHIKlwNfLhVP5?usp=sharing)\n**Finetuning example**\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1R_joYWlvToA24tsa9BFYa2D6ffiMtyVy?usp=sharing)\n\n\n### Aspect ratio images [**--\u003eNEW\u003c--**](https://github.com/shonenkov-AI/rudalle-aspect-ratio)\n\n![](https://raw.githubusercontent.com/shonenkov-AI/rudalle-aspect-ratio/main/pics/h_example.jpg)\n\n\n### [Kandinsky 12B](https://github.com/ai-forever/ru-dalle/blob/master/jupyters/Kandinsky-12b-A100.ipynb)\n\nRequest access: [Here](https://docs.google.com/forms/d/e/1FAIpQLSdYCT6LKDWgWGkd0Lq_sMLe2wZDZSkMUuCQx4qdXUnd6SrhvA/viewform)\n\n`роботы акварелью в стиле ван гога`\n![](./pics/kandinsky/example-robots.png)\n\n[![](./pics/habr_eng.svg)](https://habr.com/ru/company/sberbank/blog/671210/)\n\n![](./pics/kandinsky/loss.jpg)\n`FID = 15.4 (COCO Valid)`\n\n### 🚀 Contributors 🚀\n\n- [@bes](https://github.com/bes-dev) shared [great idea and realization with IDWT](https://github.com/bes-dev/vqvae_dwt_distiller.pytorch) for decoding images with higher quality 512x512! 😈💪 thanks a lot for your constructive advices, appreciate it\n- [@neverix](https://www.kaggle.com/neverix) thanks a lot for contributing for speed up of inference\n- [@Igor Pavlov](https://github.com/boomb0om) trained model and prepared code with [super-resolution](https://github.com/boomb0om/Real-ESRGAN-colab)\n- [@oriBetelgeuse](https://github.com/oriBetelgeuse) thanks a lot for easy API of generation using image prompt\n- [@Alex Wortega](https://github.com/AlexWortega) created first FREE version colab notebook with fine-tuning [ruDALL-E Malevich (XL)](https://huggingface.co/sberbank-ai/rudalle-Malevich) on sneakers domain 💪\n- [@Anton Lozhkov](https://github.com/anton-l) Integrated to [Huggingface Spaces](https://huggingface.co/spaces) with [Gradio](https://github.com/gradio-app/gradio), see [here](https://huggingface.co/spaces/anton-l/rudall-e)\n\n### Supported by\n\n[\u003cimg src=\"https://raw.githubusercontent.com/sberbank-ai/ru-dolph/master/pics/logo/airi-logo.png\" height=\"50\"/\u003e](https://airi.net)\n\n\n### Social Media\n\n[![](./pics/habr_eng.svg)](https://habr.com/ru/company/sberbank/blog/589673/)\n[![](./pics/habr.svg)](https://habr.com/ru/company/sberdevices/blog/586926/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fru-dalle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fai-forever%2Fru-dalle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-forever%2Fru-dalle/lists"}