https://github.com/philfung/computer-use
try Computer Use on your Mac with a few clicks
https://github.com/philfung/computer-use
anthropic claude computer-use large-language-models llms macos multimodal-large-language-models
Last synced: 6 months ago
JSON representation
try Computer Use on your Mac with a few clicks
- Host: GitHub
- URL: https://github.com/philfung/computer-use
- Owner: philfung
- Created: 2024-10-31T02:17:46.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-11-24T22:11:13.000Z (8 months ago)
- Last Synced: 2024-11-24T23:21:13.784Z (8 months ago)
- Topics: anthropic, claude, computer-use, large-language-models, llms, macos, multimodal-large-language-models
- Language: Python
- Homepage:
- Size: 105 KB
- Stars: 16
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Computer Use for MacOS 💻 
A macOS app (forked from [Anthropic's computer use demo for Ubuntu](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo)) demonstrating LLM-based computer use on your laptop.
# Caution ⚠️
Computer use is a [beta feature from Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) that has direct access to the internet and **sends screenshots of your desktop to Anthropic**.
# Running the App
### 1. Ensure you have the latest [Python 3](https://www.python.org/downloads/) (>= 3.11) and [Chrome](https://www.google.com/chrome/dr/download/) installed, and an [Anthropic API key](https://console.anthropic.com/settings/keys).
### 2. Enable *Screen Recording* permissions so that this application can take screenshots of the desktop.
* Open the *Settings* app → click on *Privacy and Security* → click on *Screen and System Recording*.
→ 
→ 
* Click the little *+* on the lower-left and add the *Terminal* application, and click on the toggle next to the app so it turns blue.
→ 
### 3. Enable *Accessibility* permissions so that this application can control your mouse and keyboard.
* Open the *Settings* app->click on *Privacy and Security*->click on *Accessibility*.
→
→ 
* Click the little *+* on the lower-left and add the *Terminal* application, and click on the toggle next to the app so it turns blue.
→
### 4. Open the *Terminal* app and run the script.
```bash
# Do this in Terminal App
git clone https://github.com/philfung/computer-use.git
cd computer-use
chmod u+x install_and_run.sh
./install_and_run.sh
```
The Computer Use app should automatically open in the Chrome browser.
### 5. Enter your [Anthropic API Key](https://console.anthropic.com/settings/keys) in the left panel of the Chrome application.
### 6. Try Computer Use by entering commands!
For example:
```bash
"Open cnn.com and click on the latest article"
"Open the Settings app and change to Dark mode"
```
# Talks
* [LLMs as Computer Users: An Overview](https://www.figma.com/deck/rsWK4sRl0dOahG59bfMhql)
# Resources
For more information, check out my [recommended reading list on computer use](https://github.com/philfung/awesome-computer-use).