{"id":13910466,"url":"https://github.com/modal-labs/devlooper","last_synced_at":"2026-01-31T19:05:31.194Z","repository":{"id":185192989,"uuid":"673039414","full_name":"modal-labs/devlooper","owner":"modal-labs","description":"A program synthesis agent that autonomously fixes its output by running tests!","archived":false,"fork":false,"pushed_at":"2024-09-19T19:28:50.000Z","size":25,"stargazers_count":462,"open_issues_count":0,"forks_count":33,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-07-18T11:39:46.014Z","etag":null,"topics":["agent","gpt-4","language-model","llm","modal","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/modal-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-07-31T18:20:32.000Z","updated_at":"2025-07-10T11:15:20.000Z","dependencies_parsed_at":"2023-08-01T02:57:22.639Z","dependency_job_id":"f41c379b-542a-4b32-99ef-1d1983b61906","html_url":"https://github.com/modal-labs/devlooper","commit_stats":null,"previous_names":["modal-labs/smol-devlooper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/modal-labs/devlooper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fdevlooper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fdevlooper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fdevlooper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fdevlooper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/modal-labs","download_url":"https://codeload.github.com/modal-labs/devlooper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/modal-labs%2Fdevlooper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28950363,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T18:30:42.805Z","status":"ssl_error","status_checked_at":"2026-01-31T18:30:19.593Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","gpt-4","language-model","llm","modal","python"],"created_at":"2024-08-07T00:01:26.239Z","updated_at":"2026-01-31T19:05:31.171Z","avatar_url":"https://github.com/modal-labs.png","language":"Python","readme":"# 🐥 devlooper\n\n`devlooper` is a program synthesis agent that autonomously fixes its output by running tests!\n\nHere's `devlooper` in action, taking 11 iterations to create a Python library that generates voronoi diagrams:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"600\" alt=\"devlooper demo\" src=\"https://github.com/modal-labs/devlooper/assets/5786378/0dfa6086-96e2-484b-92c8-23d1017471ab\"\u003e\n\u003c/p\u003e\n\n## ⚙️ How it works\n\nThis project extends [smol developer](https://github.com/smol-ai/developer) by giving it access to a [sandbox](https://modal.com/docs/guide/sandbox) to run tests in. The agent iterates until all tests pass, by updating the code and fixing the environment (installing packages).\n\n### 📦 Environment templates\n\nThe project uses environment \"templates\" to define the basic setup and test harness for a given language/framework. For now, three templates are provided:\n\n- React + Jest\n- Python\n- Rust\n\nHowever, any language/framework should work, as long as it can be installed within a container. Contributions for more templates are welcome (see [`env_templates.py`](https://github.com/modal-labs/devlooper/blob/main/src/env_templates.py)).\n\n### 🏖️ Sandbox\n\nWe use [Modal](http://modal.com/)'s new [Sandbox](https://modal.com/docs/guide/sandbox) primitive to run tests in an isolated environment and fetch the output. This allows us to construct the image incrementally as well (similar to building up a Dockerfile in layers that are cached).\n\n### 🤖 Debug loop\n\nIn each iteration, the agent runs the test command for the environment. If a non-zero exit code is received, the agent passes the `stdout` and `stderr` from the sandbox to the LLM to diagnose the error. This diagnosis is used in a separate step to generate a `DebugPlan` consisting of three types of actions:\n\n1. Inspect and fix a file\n2. Install a package in the image\n3. Run commands in the image\n\nMore types of actions can be implemented pretty easily — once again, contributions are welcome!\n\nRunning the diagnosis as a separate step seems to boost model accuracy by quite a bit (instead of immediately predicting the `DebugPlan`). We suspect the benefits are similar to why Chain-of-Thought prompting works so well.\n\n## 🧑‍🚀 Usage\n\n### Set up\n\n- Create a [Modal](http://modal.com/) account ([reach out to us](mailto:akshat@modal.com) if you are still on the waitlist!)\n- Install `modal` in your current Python environment\n\n```bash\npip install modal\n```\n\n- Create a Modal token\n\n```bash\nmodal token new\n```\n\n- Create an [OpenAI](https://openai.com/) account and get an API key\n- [Create a Modal secret](https://modal.com/secrets/create) named `openai-secret`\n\n### Generate!\n\nYou're ready to generate! From the root directory of this repo, `modal run` the program with your choice of `prompt` and `template`:\n\n```bash\nmodal run src.main --prompt=\"a simple 2D graphics library\" --template=\"rust\"\n```\n\n```bash\nmodal run src.main --prompt=\"a todo-list app\" --template=\"react\"\n```\n\n```bash\nmodal run src.main --prompt=\"a webscraper that checks if there are new reservations for a given restaurant on Resy\" --template=\"python\"\n```\n\nOnce all tests pass, the output will be written to `output/` in the same directory by default. This can be overridden using `--output-path`.\n\n## ✨ Showcase\n\n_Coming soon_\n\n## 🔮 Future directions\n\nThis project is mostly a proof of concept, and there's a lot of cool additions that will make this better. Here are some ideas:\n\n- Allowing feedback from users in the loop, or accepting an existing project + plan as input and making suggested changes to it.\n- Making the debugging prompt better with relevant parts of the code, retrieved using embeddings.\n- Go out and fetch the documentation for a package if needed.\n- Using previous edits in the prompt somewhere to prevent the model from going into a loop.\n- Synthesizing `EnvTemplate`s from scratch.\n- Generalizing this to more LLMs, including open-source ones!\n","funding_links":[],"categories":["Python","python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fdevlooper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmodal-labs%2Fdevlooper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmodal-labs%2Fdevlooper/lists"}