Claude Computer Use for Mac

Enhanced version of Anthropic Computer Use optimized for macOS with Claude 4 support, Extended Thinking, and comprehensive enhancements.

🆕 Latest Enhancements (2025)

🎯 Token Efficiency (14-70% Reduction)

Claude 3.7 Sonnet: Automatic token-efficient-tools-2025-02-19 beta header
Significant reduction in output tokens for tool use scenarios
Optimized for cost-effective long conversations

🧠 Enhanced Extended Thinking

Claude 4 Models: Support for up to 128k thinking tokens
Interleaved Thinking: interleaved-thinking-2025-05-14 beta header for Claude 4
Thinking between tool calls for sophisticated multi-step reasoning
Smart budget recommendations per model

⚡ Enhanced Computer Actions (computer_20250124)

New Actions: scroll, left_click_drag, right_click, middle_click, double_click, triple_click
Enhanced Control: left_mouse_down, left_mouse_up, hold_key, wait
Comprehensive Mac Shortcuts: 200+ built-in keyboard shortcuts with task-based aliases
Smart Key Mapping: Natural language to shortcut conversion (e.g., "copy" → "cmd+c")

🎯 Claude 4 Best Practices Integration

Explicit Instructions: Clear, systematic approach to complex tasks
Parallel Tool Execution: Optimized strategies for multi-step operations
Enhanced System Prompts: Comprehensive guidance for macOS automation
Error Recovery: Built-in fallback strategies and verification steps

🌟 Key Features

🤖 Model Support

Claude Opus 4 - Most capable model with enhanced reasoning
Claude Sonnet 4 - High performance with 64k output tokens
Claude 3.7 Sonnet - Extended thinking with token efficiency
Claude 3.5 Sonnet - Fast and reliable (legacy support)
Claude 3.5 Haiku - Rapid responses for simple tasks

🔧 Advanced Tool Capabilities

Enhanced Computer Tool: Full mouse/keyboard control with Mac-native optimizations
AppleScript Integration: High-level macOS application automation
Bash Tool: Complete command-line access with safety features
File Editor: Advanced text editing with syntax awareness
Silicon Tool: Apple Silicon performance monitoring

💻 macOS Native Integration

Apple Silicon Optimized: Native M-series chip performance
Comprehensive Shortcuts: Complete macOS keyboard shortcut database
System Automation: Native screencapture, sips, osascript integration
Mission Control: Spaces and window management support

🧠 Intelligence Features

Extended Thinking: Step-by-step reasoning for complex tasks
Smart Session Management: ChatGPT-style conversation persistence
Intelligent Auto-naming: Context-aware chat titles
Tool Usage Analytics: Performance tracking and optimization

🆕 What's New

Claude 4 Support

Claude Opus 4 - Most capable model for complex reasoning and coding
Claude Sonnet 4 - High-performance model with balanced capabilities
Up to 64k output tokens (Sonnet 4) and 32k (Opus 4)

Extended Thinking

Step-by-step reasoning for complex tasks
Configurable thinking budget (1k-32k+ tokens)
Transparent problem-solving process

Enhanced macOS Integration

AppleScript Tool - Native macOS application automation
Silicon Tool - Apple Silicon hardware monitoring
Shortcuts Integration - Run macOS Shortcuts via command line
System Profiler - Detailed hardware information

Smart Model Selection

Easy switching between Claude 3.5, 3.7, and 4 models
Automatic tool version management
Model-specific feature detection

🏗️ Architecture

Supported Models

Model	Use Case	Max Output	Tool Versions	Extended Thinking
Claude Opus 4	Most complex tasks, coding, analysis	32k tokens	`*_20250124` + `text_editor_20250429`	✅
Claude Sonnet 4	High performance, balanced tasks	64k tokens	`*_20250124` + `text_editor_20250429`	✅
Claude Sonnet 3.7	Extended thinking capabilities	64k tokens	`*_20250124` + `text_editor_20250124`	✅
Claude Sonnet 3.5 v2	Previous generation intelligent model	8k tokens	`*_20241022`	❌
Claude Haiku 3.5	Fast and efficient processing	8k tokens	`*_20241022`	❌

Tool Collection

Core Tools (Anthropic)

Computer Tool - Screen capture, mouse/keyboard control
Text Editor - File editing with str_replace operations
Bash Tool - Command line execution

Custom Tools (macOS)

AppleScript Tool - macOS application automation
Silicon Tool - Apple Silicon hardware monitoring

Tool Version Matrix

Claude 4 Models (Latest)

Computer: computer_20250124
Text Editor: text_editor_20250429 (no undo_edit)
Bash: bash_20250124
AppleScript: custom
Silicon: custom

Claude 3.7 Sonnet

Computer: computer_20250124
Text Editor: text_editor_20250124 (includes undo_edit)
Bash: bash_20250124
AppleScript: custom
Silicon: custom

Claude 3.5 Models (Legacy)

Computer: computer_20241022
Text Editor: text_editor_20241022
Bash: bash_20241022
AppleScript: custom
Silicon: custom

🚀 Quick Start

Prerequisites

macOS 10.14+ (Monterey+ recommended)
Python 3.11+
Anthropic API key
Homebrew (recommended)

Installation

Quick Setup (Recommended)

git clone https://github.com/swilliams9772/mac_computer_use.git
cd mac_computer_use

# Run the automated setup script
./setup.sh

# The script will automatically:
# - Install Homebrew (if needed)
# - Install Python 3.11+ and dependencies
# - Create virtual environment
# - Install all requirements
# - Create activation helper script

Manual Setup

git clone https://github.com/swilliams9772/mac_computer_use.git
cd mac_computer_use

# Create virtual environment
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set up API key
export ANTHROPIC_API_KEY="your-api-key-here"

# Run the Streamlit app
streamlit run app.py

# OR run the CLI version
python cli.py

CLI Usage

The CLI version provides a terminal-based interface with advanced features:

# Basic usage (interactive model selection)
python cli.py

# Specific model with extended thinking
python cli.py --model claude-3-7-sonnet-20250219 --thinking-budget 15000

# Load previous session
python cli.py --load-session

# Full configuration
python cli.py \
  --model claude-sonnet-4-20250514 \
  --thinking-budget 20000 \
  --max-tokens 8192 \
  --system-prompt "You are an expert macOS automation assistant."

CLI Features

🧠 Extended Thinking - Step-by-step reasoning display
💾 Session Persistence - Save/load conversations
⚡ Progress Indicators - Visual feedback for operations
🛠️ Error Recovery - Smart error handling with retry options
📊 Status Tracking - Real-time model and session status
🎨 Smart Output - Automatic formatting and truncation

CLI Commands

Use these during CLI sessions:

/help - Show available commands
/save - Save current session
/load - Load saved session
/clear - Clear conversation
/status - Show configuration
/exit - Exit with optional save

Environment Setup

# Install system dependencies
brew install cliclick

# Optional: Enable accessibility permissions
# System Preferences > Security & Privacy > Privacy > Accessibility
# Add Terminal and Python to allowed applications

🔧 Configuration

Model Selection

Choose your model based on your needs:

Claude Opus 4 - Complex reasoning, advanced coding, research
Claude Sonnet 4 - Balanced performance for most tasks
Claude Sonnet 3.7 - Extended thinking, complex problem solving
Claude Haiku 3.5 - Fast, lightweight tasks

Extended Thinking

When enabled, Claude will show its step-by-step reasoning:

# Configuration options
enable_extended_thinking = True
thinking_budget_tokens = 10000  # 1k-32k+ recommended

Custom Tools

The system includes custom macOS tools:

# AppleScript automation
applescript_tool.execute(
    script='display dialog "Hello from Claude!"',
    application="Finder"  # Optional target
)

# Apple Silicon monitoring
silicon_tool.monitor(
    action="performance",  # performance, thermal, memory, system_info
    target=None
)

Project Structure

mac_computer_use/
├── tools/                           # Tool implementations
│   ├── __init__.py                  # Tool exports
│   ├── base.py                      # Base tool classes
│   ├── collection.py                # Tool collection manager
│   ├── computer.py                  # Computer use tool (mouse/keyboard/screen)
│   ├── edit.py                      # Text editor tool
│   ├── bash.py                      # Bash execution tool
│   ├── applescript.py               # AppleScript automation (custom)
│   ├── silicon.py                   # Apple Silicon monitoring (custom)
│   └── run.py                       # Command execution utilities
├── anthropic_docs_md/               # Anthropic documentation reference
├── loop.py                          # Main sampling loop
├── app.py                           # Streamlit web UI
├── cli.py                           # CLI interface
├── setup.sh                         # Automated setup script
├── requirements.txt                 # Python dependencies
├── Dockerfile                       # Container configuration
├── .gitignore                       # Git ignore rules
├── LICENSE                          # MIT License
└── README.md                        # This file

🎯 Usage Examples

Basic Computer Use

# Take a screenshot and analyze
# Claude can see the screen and interact with applications

# Navigate to a website
# Claude can control mouse and keyboard

# Edit files
# Claude can read, write, and modify files

AppleScript Automation

# Control macOS applications
applescript_tool.execute(
    script='''
    tell application "Safari"
        make new document
        set URL of current tab to "https://anthropic.com"
    end tell
    ''',
    application="Safari"
)

System Monitoring

# Check Apple Silicon performance
silicon_tool.monitor(action="performance")

# Monitor thermal status
silicon_tool.monitor(action="thermal")

# Check memory usage
silicon_tool.monitor(action="memory")

🔐 Security & Privacy

Screen Access - Required for computer tool functionality
Accessibility - Needed for mouse/keyboard control
API Keys - Stored locally, never transmitted except to Anthropic
Local Execution - All tools run locally on your machine

🐛 Troubleshooting

Common Issues

Tool Type Validation Error
```
Input tag 'applescript_20250124' found using 'type' does not match any of the expected tags
```
Solution: Custom tools now use type: "custom" - this is fixed in the latest version.
Extended Thinking Not Available
```
Model does not support extended thinking
```
Solution: Use Claude 3.7, Sonnet 4, or Opus 4 models.
Max Tokens Exceeded
```
prompt tokens + max_tokens exceeds context window
```
Solution: Reduce max_tokens or enable image filtering.

Performance Tips

Use Claude Haiku 3.5 for fast, simple tasks
Enable Extended Thinking for complex reasoning
Set appropriate thinking budget (10k default, 32k+ for complex tasks)
Use image filtering to manage context window

🛠️ Development

Adding Custom Tools

Create tool class in tools/ directory
Inherit from BaseAnthropicTool
Set api_type = "custom"
Implement __call__ and to_params methods
Add to ToolCollection in loop.py

Testing

# Run basic tests
python -m pytest tests/

# Test specific tool
python -c "from tools import AppleScriptTool; print('✅ Import successful')"

# Verify API connectivity
python -c "import anthropic; print('✅ Anthropic SDK ready')"

📋 Requirements

System Requirements

macOS 10.14+ (Monterey+ recommended for full features)
Python 3.11+
8GB+ RAM (16GB+ recommended for extended thinking)
Internet connection for API calls

Python Dependencies

See requirements.txt for complete list:

anthropic>=0.40.0
streamlit>=1.40.0
pillow>=10.0.0
And others...

Contributing

Contributions are welcome! Please read CONTRIBUTING.md for guidelines on how to submit pull requests, report issues, and contribute to the project.

License

This project is licensed under the MIT License.

Acknowledgments

Anthropic for Claude and Computer Use tools
Original Computer Use Demo
Mac Computer Use fork
macOS and Apple Silicon optimization community

Report a Bug · Request a Feature

Name		Name	Last commit message	Last commit date
Latest commit History 26 Commits
.github		.github
anthropic_docs_md		anthropic_docs_md
tools		tools
.dockerignore		.dockerignore
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
LICENSE		LICENSE
MAC_SHORTCUTS_README.md		MAC_SHORTCUTS_README.md
PERFORMANCE_IMPROVEMENTS.md		PERFORMANCE_IMPROVEMENTS.md
README.md		README.md
SECURITY.md		SECURITY.md
__init__.py		__init__.py
app.py		app.py
aws_setup_guide.md		aws_setup_guide.md
bedrock-iam-policy.json		bedrock-iam-policy.json
cli.py		cli.py
loop.py		loop.py
requirements.txt		requirements.txt
setup.sh		setup.sh

Folders and files

Latest commit

History

Repository files navigation

Claude Computer Use for Mac

🆕 Latest Enhancements (2025)

🎯 Token Efficiency (14-70% Reduction)

🧠 Enhanced Extended Thinking

⚡ Enhanced Computer Actions (computer_20250124)

🎯 Claude 4 Best Practices Integration

🌟 Key Features

🤖 Model Support

🔧 Advanced Tool Capabilities

💻 macOS Native Integration

🧠 Intelligence Features

🆕 What's New

Claude 4 Support

Extended Thinking

Enhanced macOS Integration

Smart Model Selection

🏗️ Architecture

Supported Models

Tool Collection

Core Tools (Anthropic)

Custom Tools (macOS)

Tool Version Matrix

Claude 4 Models (Latest)

Claude 3.7 Sonnet

Claude 3.5 Models (Legacy)

🚀 Quick Start

Prerequisites

Installation

Quick Setup (Recommended)

Manual Setup

CLI Usage

CLI Features

CLI Commands

Environment Setup

🔧 Configuration

Model Selection

Extended Thinking

Custom Tools

Project Structure

🎯 Usage Examples

Basic Computer Use

AppleScript Automation

System Monitoring

🔐 Security & Privacy

🐛 Troubleshooting

Common Issues

Performance Tips

🛠️ Development

Adding Custom Tools

Testing

📋 Requirements

System Requirements

Python Dependencies

Contributing

License

Acknowledgments

About

Topics

Resources

License

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages