{"id":25649556,"url":"https://github.com/e2b-dev/computer-use-app","last_synced_at":"2025-04-15T18:20:04.268Z","repository":{"id":277677154,"uuid":"928453522","full_name":"e2b-dev/computer-use-app","owner":"e2b-dev","description":"A web playground for a secure and open source computer use. Powered by E2B.","archived":false,"fork":false,"pushed_at":"2025-02-21T16:31:39.000Z","size":575,"stargazers_count":9,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-22T18:54:49.379Z","etag":null,"topics":["ai","computer-use","llama3","open-source","qwen"],"latest_commit_sha":null,"homepage":"https://computer-use-app.vercel.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/e2b-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-06T17:04:44.000Z","updated_at":"2025-02-21T16:31:43.000Z","dependencies_parsed_at":"2025-02-15T11:29:34.857Z","dependency_job_id":"f4a1dc5e-889d-4bdc-adf9-b7025ba7d70c","html_url":"https://github.com/e2b-dev/computer-use-app","commit_stats":null,"previous_names":["e2b-dev/computer-use-app"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e2b-dev%2Fcomputer-use-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e2b-dev%2Fcomputer-use-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e2b-dev%2Fcomputer-use-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/e2b-dev%2Fcomputer-use-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/e2b-dev","download_url":"https://codeload.github.com/e2b-dev/computer-use-app/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240325157,"owners_count":19783623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","computer-use","llama3","open-source","qwen"],"created_at":"2025-02-23T14:22:32.114Z","updated_at":"2025-02-23T14:22:32.962Z","avatar_url":"https://github.com/e2b-dev.png","language":"TypeScript","readme":"# Computer Use App\n\nThis application allows you to interact with a remote desktop environment using natural language. It leverages the E2B desktop environment and AI models to execute commands and automate tasks.\n\n\nhttps://github.com/user-attachments/assets/9f1a22ff-eba4-451f-adec-d9cc3c9763aa\n\n\n## Prerequisites\n\nBefore starting, you'll need:\n\n1. [Node.js](https://nodejs.org/) 18 or later\n2. [npm](https://www.npmjs.com/) (comes with Node.js)\n3. An [E2B API key](https://e2b.dev/docs/getting-started/api-key)\n4. One of the following AI model API keys:\n   - [Anthropic API key](https://console.anthropic.com/) for Claude 3.5 Sonnet\n   - [OpenAI API key](https://platform.openai.com/api-keys) for GPT-4o\n   - [Google API key](https://aistudio.google.com/apikey) for Gemini 2.0 Flash\n   - [XAI API key](https://console.x.ai/) for Grok 2 Vision\n   - [Mistral API key](https://console.mistral.ai/) for Mistral Large\n   - [Groq API key](https://console.groq.com/) for Llama 3.3 70B\n\n## Setup Instructions\n\n1. **Clone the repository**\n```bash\ngit clone https://github.com/e2b-dev/computer-use-app\ncd computer-use-app\n```\n\n2. **Install dependencies**\n```bash\nnpm install\n```\n\n3. **Set up environment variables**\n\nCreate a `.env.local` file in the root directory:\n\n```env\n# Required: E2B API key\nE2B_API_KEY=your_e2b_api_key\n\n# Optional: AI Model API keys (at least one is required)\n# Choose the model(s) you want to use:\n\n# For Claude 3.5 Sonnet (recommended default)\nANTHROPIC_API_KEY=your_anthropic_api_key\n\n# For GPT-4o (alternative)\nOPENAI_API_KEY=your_openai_api_key\n\n# For Gemini 2.0 Flash (alternative)\nGOOGLE_API_KEY=your_google_api_key\n\n# For Grok 2.0 (alternative)\nXAI_API_KEY=your_grok_api_key\n\n# For Mistral Large (alternative)\nMISTRAL_API_KEY=your_mistral_api_key\n\n# For Llama 3.3 70B (alternative)\nGROQ_API_KEY=your_groq_api_key\n```\n\nNote: \n- The E2B API keys are required for the desktop environment to work\n- You need at least one AI model API key, but you don't need all of them\n- Claude 3.5 Sonnet is the recommended default model due to its better grounding capabilities\n\n4. **Start the development server**\n```bash\nnpm run dev\n```\n\n5. **Open the application**\n\nNavigate to [http://localhost:3000](http://localhost:3000) in your browser.\n\n## Features\n\n- **Autonomous Desktop AI Agent:** An AI agent that can interact with a remote desktop environment using natural language commands.\n- **AI Model Integration:** Supports multiple AI models:\n  - Claude 3.5 Sonnet (Anthropic)\n  - GPT-4o (OpenAI)\n  - Gemini 2.0 Flash (Google)\n  - Grok 2 Vision (XAI)\n  - Mistral Large (Mistral)\n  - Llama 3.3 70B (Groq)\n- **Tool Execution:** Executes bash commands and simulates mouse/keyboard interactions.\n- **UI Framework:** Next.js, Tailwind CSS, and shadcn/ui for building the user interface.\n\n## Usage\n\n1. Click \"Start Instance\" to initialize the remote desktop environment\n2. Select your preferred AI model from the dropdown\n3. Type your instruction in the chat input (e.g., \"open Firefox and go to google.com\")\n4. Watch as the AI executes your commands in the desktop stream\n\n## Model Capabilities\n\n| Model | Vision | Action | Grounding |\n|-------|---------|---------|------------|\n| Claude 3.5 Sonnet | ✅ | ✅ | ✅ |\n| GPT-4o | ✅ | ✅ | ShowUI and OS Atlas |\n| Gemini 2.0 Flash | ✅ | ✅ | ShowUI and OS Atlas |\n| Grok 2.0 | ✅ | ✅ | ShowUI and OS Atlas |\n| Mistral Large | Pixtral Large | ✅ | ShowUI and OS Atlas |\n| Llama 3.3 70B | Llama 3.2 11B Vision | ✅ | ShowUI and OS Atlas |\n\n## Troubleshooting\n\n- **Sandbox not starting**: Verify your E2B API key is correct in `.env.local`\n- **Model not responding**: Check that you've set up the corresponding API key for your selected model\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\n## Support\n\nIf you encounter any issues or have questions:\n- Check the [E2B Documentation](https://e2b.dev/docs)\n- Join the [E2B Discord](https://discord.gg/U7KEcGErtQ)\n- Open an [issue](https://github.com/e2b-dev/computer-use-app/issues)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe2b-dev%2Fcomputer-use-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fe2b-dev%2Fcomputer-use-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fe2b-dev%2Fcomputer-use-app/lists"}