{"id":13422570,"url":"https://github.com/lucidrains/big-sleep","last_synced_at":"2025-05-14T15:10:18.418Z","repository":{"id":37243769,"uuid":"330768122","full_name":"lucidrains/big-sleep","owner":"lucidrains","description":"A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun","archived":false,"fork":false,"pushed_at":"2022-02-06T18:04:34.000Z","size":7221,"stargazers_count":2572,"open_issues_count":62,"forks_count":303,"subscribers_count":47,"default_branch":"main","last_synced_at":"2025-04-11T05:13:29.438Z","etag":null,"topics":["artificial-intelligence","deep-learning","generative-adversarial-networks","multimodality","text-to-image"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucidrains.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-18T19:35:36.000Z","updated_at":"2025-03-26T01:35:32.000Z","dependencies_parsed_at":"2022-07-10T14:01:07.543Z","dependency_job_id":null,"html_url":"https://github.com/lucidrains/big-sleep","commit_stats":null,"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fbig-sleep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fbig-sleep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fbig-sleep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucidrains%2Fbig-sleep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucidrains","download_url":"https://codeload.github.com/lucidrains/big-sleep/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248345268,"owners_count":21088245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","deep-learning","generative-adversarial-networks","multimodality","text-to-image"],"created_at":"2024-07-30T23:00:47.982Z","updated_at":"2025-04-11T05:13:33.906Z","avatar_url":"https://github.com/lucidrains.png","language":"Python","funding_links":[],"categories":["Interactive Learning \u0026 Tools","Python","🎨 Creative AI","Applications"],"sub_categories":["Text to Image AI's","Text-to-Image Generation"],"readme":"\u003cimg src=\"./samples/artificial_intelligence.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*artificial intelligence*\n\n\u003cimg src=\"./samples/cosmic_love_and_attention.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*cosmic love and attention*\n\n\u003cimg src=\"./samples/fire_in_the_sky.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*fire in the sky*\n\n\u003cimg src=\"./samples/a_pyramid_made_of_ice.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*a pyramid made of ice*\n\n\u003cimg src=\"./samples/a_lonely_house_in_the_woods.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*a lonely house in the woods*\n\n\u003cimg src=\"./samples/marriage_in_the_mountains.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*marriage in the mountains*\n\n\u003cimg src=\"./samples/a_lantern_dangling_from_the_tree_in_a_foggy_graveyard.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*lantern dangling from a tree in a foggy graveyard*\n\n\u003cimg src=\"./samples/a_vivid_dream.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*a vivid dream*\n\n\u003cimg src=\"./samples/balloons_over_the_ruins_of_a_city.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*balloons over the ruins of a city*\n\n\u003cimg src=\"./samples/the_death_of_the_lonesome_astronomer.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*the death of the lonesome astronomer* - by \u003ca href=\"https://github.com/moirage\"\u003emoirage\u003c/a\u003e\n\n\u003cimg src=\"./samples/the_tragic_intimacy_of_the_eternal_conversation_with_oneself.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*the tragic intimacy of the eternal conversation with oneself* - by \u003ca href=\"https://github.com/moirage\"\u003emoirage\u003c/a\u003e\n\n\u003cimg src=\"./samples/demon_fire.png\" width=\"250px\"\u003e\u003c/img\u003e\n\n*demon fire* - by \u003ca href=\"https://github.com/WiseNat\"\u003eWiseNat\u003c/a\u003e\n\n## Big Sleep\n\n\u003ca href=\"https://twitter.com/advadnoun\"\u003eRyan Murdock\u003c/a\u003e has done it again, combining OpenAI's \u003ca href=\"https://github.com/openai/CLIP\"\u003eCLIP\u003c/a\u003e and the generator from a \u003ca href=\"https://arxiv.org/abs/1809.11096\"\u003eBigGAN\u003c/a\u003e! This repository wraps up his work so it is easily accessible to anyone who owns a GPU.\n\nYou will be able to have the GAN dream up images using natural language with a one-line command in the terminal.\n\nOriginal notebook [![Open In Colab][colab-badge]][colab-notebook]\n\nSimplified notebook [![Open In Colab][colab-badge]][colab-notebook-2]\n\nUser-made notebook with bugfixes and added features, like google drive integration [![Open In Colab][colab-badge]][user-made-colab-notebook]\n\n[user-made-colab-notebook]: \u003chttps://colab.research.google.com/drive/1zVHK4t3nXQTsu5AskOOOf3Mc9TnhltUO?usp=sharing\u003e\n[colab-notebook]: \u003chttps://colab.research.google.com/drive/1NCceX2mbiKOSlAd_o7IU7nA9UskKN5WR?usp=sharing\u003e\n[colab-notebook-2]: \u003chttps://colab.research.google.com/drive/1MEWKbm-driRNF8PrU7ogS5o3se-ePyPb?usp=sharing\u003e\n[colab-badge]: \u003chttps://colab.research.google.com/assets/colab-badge.svg\u003e\n\n## Install\n\n```bash\n$ pip install big-sleep\n```\n\n## Usage\n\n```bash\n$ dream \"a pyramid made of ice\"\n```\n\nImages will be saved to wherever the command is invoked\n\n## Advanced\n\nYou can invoke this in code with\n\n```python\nfrom big_sleep import Imagine\n\ndream = Imagine(\n    text = \"fire in the sky\",\n    lr = 5e-2,\n    save_every = 25,\n    save_progress = True\n)\n\ndream()\n```\n\n\u003e You can now train more than one phrase using the delimiter \"|\"\n\n### Train on Multiple Phrases\nIn this example we train on three phrases:\n\n- `an armchair in the form of pikachu` \n- `an armchair imitating pikachu`\n- `abstract`\n\n```python\nfrom big_sleep import Imagine\n\ndream = Imagine(\n    text = \"an armchair in the form of pikachu|an armchair imitating pikachu|abstract\",\n    lr = 5e-2,\n    save_every = 25,\n    save_progress = True\n)\n\ndream()\n```\n\n### Penalize certain prompts as well!\n\nIn this example we train on the three phrases from before,\n\n**and** *penalize* the phrases:\n- `blur`\n- `zoom`\n```python\nfrom big_sleep import Imagine\n\ndream = Imagine(\n    text = \"an armchair in the form of pikachu|an armchair imitating pikachu|abstract\",\n    text_min = \"blur|zoom\",\n)\ndream()\n```\n\n\nYou can also set a new text by using the `.set_text(\u003cstr\u003e)` command\n\n```python\ndream.set_text(\"a quiet pond underneath the midnight moon\")\n```\n\nAnd reset the latents with `.reset()`\n\n```python\ndream.reset()\n```\n\nTo save the progression of images during training, you simply have to supply the `--save-progress` flag\n\n```bash\n$ dream \"a bowl of apples next to the fireplace\" --save-progress --save-every 100\n```\n\nDue to the class conditioned nature of the GAN, Big Sleep often steers off the manifold into noise. You can use a flag to save the best high scoring image (per CLIP critic) to `{filepath}.best.png` in your folder.\n\n```bash\n$ dream \"a room with a view of the ocean\" --save-best\n```\n\n## Larger model\n\nIf you have enough memory, you can also try using a bigger vision model released by OpenAI for improved generations.\n\n```bash\n$ dream \"storm clouds rolling in over a white barnyard\" --larger-model\n```\n\n## Experimentation\n\nYou can set the number of classes that you wish to restrict Big Sleep to use for the Big GAN with the `--max-classes` flag as follows (ex. 15 classes). This may lead to extra stability during training, at the cost of lost expressivity.\n\n```bash\n$ dream 'a single flower in a withered field' --max-classes 15\n```\n\n## Alternatives\n\n\u003ca href=\"https://github.com/lucidrains/deep-daze\"\u003eDeep Daze\u003c/a\u003e - CLIP and a deep SIREN network\n\n## Citations\n\n```bibtex\n@misc{unpublished2021clip,\n    title  = {CLIP: Connecting Text and Images},\n    author = {Alec Radford, Ilya Sutskever, Jong Wook Kim, Gretchen Krueger, Sandhini Agarwal},\n    year   = {2021}\n}\n```\n\n```bibtex\n@misc{brock2019large,\n    title   = {Large Scale GAN Training for High Fidelity Natural Image Synthesis}, \n    author  = {Andrew Brock and Jeff Donahue and Karen Simonyan},\n    year    = {2019},\n    eprint  = {1809.11096},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.LG}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fbig-sleep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucidrains%2Fbig-sleep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucidrains%2Fbig-sleep/lists"}