https://github.com/philfung/computer-use

macOS app demonstrating Claude's LLM-based computer use
https://github.com/philfung/computer-use

anthropic claude computer-use large-language-models llms macos multimodal-large-language-models

Last synced: about 2 months ago
JSON representation

macOS app demonstrating Claude's LLM-based computer use

• Host: GitHub
• URL: https://github.com/philfung/computer-use
• Owner: philfung
• Created: 2024-10-31T02:17:46.000Z (2 months ago)
• Default Branch: main
• Last Pushed: 2024-11-08T02:58:10.000Z (about 2 months ago)
• Last Synced: 2024-11-08T03:38:17.850Z (about 2 months ago)
• Topics: anthropic, claude, computer-use, large-language-models, llms, macos, multimodal-large-language-models
• Language: Python
• Homepage:
• Size: 80.1 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Computer Use for MacOS 💻

A macOS app (forked from [Anthropic's computer use demo for Ubuntu](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo)) demonstrating LLM-based computer use on your laptop.



# Caution ⚠️

Computer use is a [beta feature from Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) that has direct access to the internet and sends screenshots of your desktop to Anthropic. 

# Running the App

### 1. Ensure you have the latest [Python 3](https://www.python.org/downloads/) (>= 3.11) and [Chrome](https://www.google.com/chrome/dr/download/) installed, and an [Anthropic API key](https://console.anthropic.com/settings/keys).

### 2. Enable *Screen Recording* permissions so that this application can take screenshots of the desktop.

* Open the *Settings* app → click on *Privacy and Security* → click on *Screen and System Recording*.

  

→ 

→ 

* Click the little *+* on the lower-left and add the *Terminal* application, and click on the toggle next to the app so it turns blue.

  

→  

### 3. Enable *Accessibility* permissions so that this application can control your mouse and keyboard.  

* Open the *Settings* app->click on *Privacy and Security*->click on *Accessibility*.

  

→ 

→ 

* Click the little *+* on the lower-left and add the *Terminal* application, and click on the toggle next to the app so it turns blue.

  

→  

### 4. Open the *Terminal* app and run the script.

```bash

# Do this in Terminal App

git clone https://github.com/philfung/computer-use.git

cd computer-use

chmod u+x install_and_run.sh

./install_and_run.sh

```

  The Computer Use app should automatically open in the Chrome browser.

    

### 5. Enter your [Anthropic API Key](https://console.anthropic.com/settings/keys) in the left panel of the Chrome application.

    

### 6. Try Computer Use by entering commands!

For example:

```bash

"Open cnn.com and click on the latest article"

"Open the Settings app and lower the screen brightness"

```