{"id":13763866,"url":"https://github.com/minimaxir/ctrl-gce","last_synced_at":"2025-04-14T15:41:00.984Z","repository":{"id":65982385,"uuid":"208499788","full_name":"minimaxir/ctrl-gce","owner":"minimaxir","description":"Set up the CTRL text-generating model on Google Compute Engine with just a few console commands.","archived":false,"fork":false,"pushed_at":"2019-10-26T11:04:24.000Z","size":134,"stargazers_count":154,"open_issues_count":3,"forks_count":13,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-05-01T21:40:52.746Z","etag":null,"topics":["ctrl","google-compute-engine","text-generation"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/minimaxir.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"minimaxir","patreon":"minimaxir","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2019-09-14T20:28:26.000Z","updated_at":"2024-05-01T21:40:52.747Z","dependencies_parsed_at":"2023-02-19T19:00:49.254Z","dependency_job_id":null,"html_url":"https://github.com/minimaxir/ctrl-gce","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/minimaxir%2Fctrl-gce","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/minimaxir%2Fctrl-gce/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/minimaxir%2Fctrl-gce/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/minimaxir%2Fctrl-gce/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/minimaxir","download_url":"https://codeload.github.com/minimaxir/ctrl-gce/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248907892,"owners_count":21181425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ctrl","google-compute-engine","text-generation"],"created_at":"2024-08-03T15:01:00.852Z","updated_at":"2025-04-14T15:41:00.964Z","avatar_url":"https://github.com/minimaxir.png","language":"Shell","funding_links":["https://github.com/sponsors/minimaxir","https://patreon.com/minimaxir","https://www.patreon.com/minimaxir"],"categories":["Text Generation/NLP","Shell"],"sub_categories":["Other"],"readme":"# Install and Use CTRL on Google Compute Engine\n\n![](demo.png)\n\nScripts + guides on how to set up a virtual machine on [Google Compute Engine](https://cloud.google.com/compute/) capable of running and using [Salesforce](https://www.salesforce.com)'s very large text-generating model [CTRL](https://github.com/salesforce/ctrl) to generate high-quality text based on conditional parameters.\n\nThe CTRL model is so large (12 GB on disk, 15.5 GB GPU VRAM when loaded, even more system RAM during runtime) that it will currently not fit into a free Colaboratory or Kaggle Notebook. Therefore, this setup is necessary to play with the model for now.\n\n## Machine Setup Instructions\n\nThe VM these instructions create is the minimum, lowest-cost configuration powerful enough to run CTRL without going out-of-memory (P100 GPU, 8 vCPU, 30 GB RAM, preemptible). With this configuration, having the VM up will cost **$0.51/hr**.\n\n1. Make sure the `gcloud` command line tool is [set up](https://cloud.google.com/sdk/gcloud/) on your local computer and up-to-date (can update via `gcloud components update`).\n2. Make sure your Google Cloud Platform project tied to your local computer's `gcloud` has enough [quota](https://console.cloud.google.com/iam-admin/quotas) in the `us-central-1` region (8 CPUs and 1 P100; these should be available by default, but request more quota if they aren't)\n3. Make sure your GCE project has billing set up.\n4. On your local computer, run this `gcloud` command in a terminal which creates a VM with the specs noted above:\n\n```sh\ngcloud compute instances create ctrl \\\n  --zone us-central1-c \\\n  --boot-disk-size 45GB \\\n  --image-family tf-latest-gpu \\\n  --image-project deeplearning-platform-release \\\n  --maintenance-policy TERMINATE \\\n  --machine-type n1-standard-8 \\\n  --metadata install-nvidia-driver=True \\\n  --preemptible \\\n  --accelerator='type=nvidia-tesla-p100,count=1'\n  \n```\n\nYou can view the created instance, Start/Stop it, and delete it, in the [Google Compute Engine](https://console.cloud.google.com/compute/instances) dashboard.\n\nOnce created (after waiting a bit for the GPU drivers to install), SSH into the instance. The recommended way to do is via the `gcloud` command created from the SSH dropdown in the GCE dashboard, which will look like:\n\n```sh\ngcloud beta compute --project \"\u003cPROJECT ID\u003e\" ssh --zone \"us-central1-c\" \"ctrl\"\n```\n\nWhile SSHed into the VM you created, download and run the `install_gce.sh` [script](https://github.com/minimaxir/ctrl-gce/blob/master/install_gce.sh) from this repo via:\n\n```sh\ncurl -O -s https://raw.githubusercontent.com/minimaxir/ctrl-gce/master/install_gce.sh\nsudo sh install_gce.sh\n```\n\nIf you'd like to use the model with a sequence length of 512, you can pass `512` as an argument to the install script, e.g.\n\n```sh\nsudo sh install_gce.sh 512\n```\n\nYou're done! Make sure to Stop the instance in the GCE dashboard when you are finished generating text!\n\n## Using CTRL\n\nFor basic usage, running the command below in the VM will load the model and eventually start an interactive prompt.\n\n```sh\nsudo python generation.py --model_dir seqlen256_v1.ckpt/\n```\n\nIf you are using the 512-length model, instead you would do:\n\n```sh\nsudo python generation.py --model_dir seqlen512_v1.ckpt/ --generate_num 512\n```\n\nWhile generating, you can specify a KeyboardInterrupt to stop generation (Ctrl+C on macOS). It's also recommended to clear the terminal (CMD+K on macOS) occasionally as the entire generated text will be output after each added token.\n\nYou **must** include a control code with each interactive prompt. You can see how the control codes are used in the original paper, or refer to the following examples:\n\n### Links\n\nThe `Links` control code allows you to specify a URL and/or a prompt text and have the model attempt to extrapolate the corresponding article. Examples:\n\n`Links https://www.buzzfeednews.com/article/annehelenpetersen/jeremy-renner-app-trolling-career-hurt-locker-avengers`\n\n```\nThe actor who played the villain Loki in Avengers: Infinity War has been accused of being a troll. \n \nActor Jeremy Renner was criticized for his role as Loki, an antagonist from Marvel's upcoming Avengers movie, after he tweeted that fans should \"stop trying to make me look bad.\" \n \nIn response to Renner's tweet, many people on Twitter said they were offended by what they perceived as trolling.\n```\n\n`Links https://www.nytimes.com/2020/08/29/us/politics/skynet-ai-online-norad.html`\n\n```\nSite Mobile Navigation \n \nA New Online Course for the Super-Smart \n \nThe University of Washington is offering a new online course on artificial intelligence that will be taught by an expert in machine learning and neural networks. \n \nThe course, called “Skynet,” was developed with help from Google, Facebook, Microsoft and other companies. It has been designed to teach students how to use deep learning, which uses computer programs modeled after neurons to analyze large amounts of data — like images or videos — using only basic rules about what each pixel represents. \n \nIn addition to teaching them how to build their own programs, it also teaches them how to apply those algorithms to real-world problems. \n```\n\n### Questions\n\nThe `Questions` control code allows you to specify a question; CTRL will attempt to follow up from that question (your mileage will vary). Examples:\n\n`Questions Q: Who is the Zodiac Killer?`\n\n```\nA: charles manson \n```\n\n`Questions Q: What is the last digit of pi?`\n\n```\nA: 0\n```\n\n### Reddit Subreddits\n\nEach Reddit subreddit has a different control code; see the [bottom of the paper](https://einstein.ai/presentations/ctrl.pdf) for a full list. You can optionally specify the Title or the self-post Text body. Examples:\n\n`Legal Title: [FL] Is it legal to get into legal trouble?`\n\n```\nText: I was driving home from work and saw a police car pull up behind me. The officer asked me for my license and registration. He then proceeded to ask if i had any drugs in the car. \n \nMy question is this: \n \n1. Can you be arrested without probable cause or even being charged with anything at all? \n \n2. If so, what are your chances of getting out of this situation? \n \n3. What happens when they arrest you? \n \n4. How do you go about proving that there were no drugs? \n \n5. Do you have an attorney who can help me? \n \nThanks \n \nEdit: \n \nThank you all so much for your responses. This has been very helpful as I'm trying to figure out how best to proceed. \n```\n\n`Opinion Text: Recycling is bad for the world.`\n\n```\nIt takes a lot of energy to recycle and it creates more waste than it saves. \n \nI think that recycling should be banned because we are destroying our planet with it. \n \nEdit: Thanks everyone who has given me some good arguments against recycling. \n \nEdit 2: I'm not saying that people shouldn't recycle; it's just that they need to do so in an environmentally friendly way.\n```\n\n### Amazon Reviews\n\nThe `Reviews` tag can be used to generate Amazon reviews at an optional specified rating or starting text.\n\n`Reviews Rating: 1.0`\n\n```\nI bought this book because it was recommended by a friend. It is not worth the money. The author has no credentials and his writing style is very poor.\n```\n\n`Reviews Rating: 5.0\\n\\nI died`\n\n```\na little inside when I saw the first page of this book. It was so beautiful and it made me feel like I could do anything. But then I read on to see what happened next. And there were no more pages. The book just stopped. No epilogue, nothing. Just an abrupt ending. I'm not sure if it's because there's another one coming out or what, but that's how I feel. It's almost as though she got tired of writing about her life in New York City and decided that she'd write something else instead.\n```\n\n### Command Line Arguments\n\nUnlike other text-generating apps, CTRL has a default `temperature` of 0, meaning the model chooses the best guess when possible (before repetition `penalty` is applied). Some CLI arguments you can add:\n\n* `--generate_num` — Number of tokens to generate (default: 256, can exceed the model window)\n* `--temperature` — Controls model creativity (default: 0, may want to increase to 0.2)\n* `--nucleus` — Controls cutoff for nucleus/top-*p* sampling (default: 0, may want to set to 0.9)\n* `--topk` — Controls cutoff for top-*k* sampling (default: 0)\n* `--penalty` — Repetition penalty factor (default: 1.2)\n\n## Notes\n\n* Since the model is huge, generation is *very* slow: about 2 tokens per second with the configuration above. (therefore, it takes about 2 minutes for a full generation with default parameters)\n* The BPEs CTRL uses are \"longer\" that those used in GPT-2. As a result, a 256-token generation in CTRL is about the same decoded length as a 1024-token generation in GPT-2.\n* When using the `Links` control code, keep in mind that code is conditioned on [OpenWebText](https://github.com/jcpeterson/openwebtext), which is conditioned on Reddit data. Therefore, there's a bias toward English websites and Reddit-friendly content. [Here's a quick spreadsheet](https://docs.google.com/spreadsheets/d/1O3_1RTnGn_ozcvf6eDLpsmf8ElxIqHg0FKbVfDxGS88/edit?usp=sharing) of the most popular domains on Reddit, sans some obvious image-oriented websites.\n* If CTRL gets confused by the `Links` URL, it tends to fall back to a more general news-oriented output.\n* It is recommended to use Google Compute Engine (even if you aren't following this guide) as the model itself is hosted in Google Cloud Storage and thus it's relatively fast to transfer to a VM (\u003e100 Mb/s), and also lowers the cost for Salesforce.\n\n## TODO\n\n* Support/Test domain detection.\n\n## Maintainer/Creator\n\nMax Woolf ([@minimaxir](https://minimaxir.com))\n\n*Max's open-source projects are supported by his [Patreon](https://www.patreon.com/minimaxir). If you found this project helpful, any monetary contributions to the Patreon are appreciated and will be put to good creative use.*\n\n## Special Thanks\n\n[Adam King](https://twitter.com/AdamDanielKing) for identifying a working implementation of loading the model after unexplained setbacks.\n\n## License\n\nMIT\n\n## Disclaimer\n\nThis repo has no affiliation or relationship with the CTRL team and/or Salesforce.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fminimaxir%2Fctrl-gce","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fminimaxir%2Fctrl-gce","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fminimaxir%2Fctrl-gce/lists"}