Clawdcursor

What it does

Clawd Cursor provides an OS-level desktop automation server, giving AI models the ability to interact with a real computer. It allows any AI model – including Claude, GPT, Gemini, and Llama – to perform tasks like moving the mouse, capturing screenshots, and interacting with applications. This is achieved through a model-agnostic architecture that works seamlessly with tool-calling models, even when using local models.

When to use it

• Automated data extraction: Have an AI agent extract specific information from web pages or documents displayed on your screen.
• Application control: Allow an AI to automate repetitive tasks within software applications like spreadsheets or design tools.
• Remote assistance: Enable an AI to remotely interact with a user's computer for troubleshooting or support purposes.
• Screen recording and analysis: Use the AI to capture screen recordings and analyze them for specific events or patterns.

Key capabilities

• Model-agnostic compatibility (Claude, GPT, Gemini, Llama)
• Desktop automation via mouse movement and clicks
• Screenshot capturing functionality
• Works with local AI models
• macOS native host app for simplified permissions management
• Localhost IPC architecture for enhanced security

Example prompts

• "Move the mouse to the top-left corner of the screen and click."
• "Capture a screenshot of the current window and save it as 'screenshot.png'."
• “Drag the mouse from this coordinate (x, y) to that coordinate (a, b).”

Tips & gotchas

• On macOS, you'll need to grant Accessibility and Screen Recording permissions to the Clawd Cursor host application for full functionality.
• The system utilizes a token-based authentication method via ~/.clawdcursor/host-token which should be kept secure (mode 0600).