How to Give Your OpenClaw Agent Full macOS UI Control with Peekaboo

A common question from users building autonomous agents: How do I give my OpenClaw agent complete control over my Mac — seeing the screen and navigating apps, not just browser automation?

Based on a recent Discord thread, here's how to set up full macOS UI control using Peekaboo.

The Two Capabilities

Unlike the browser tool which works against a DOM, macOS desktop control requires different tools:

1. Screen Access (Capture)

Screenshots + UI mapping: via the peekaboo CLI/skill
Screen video: via Nodes → screen.record (returns mp4)

2. UI Control (Click/Type/Navigate)

Peekaboo UI automation: brokered through the macOS companion app using PeekabooBridge
Supports: clicking, typing, window management, menu navigation, dock actions

Setup Guide (Mac mini Full Control)

Step 1: Install OpenClaw.app (macOS Companion)

The companion app is required for UI automation. It doesn't replace your Gateway — it coexists with it.

Common concern: "Won't installing the app create another gateway?"

No! In Local mode (default), OpenClaw.app attaches to your running Gateway. It only manages permissions and Peekaboo Bridge.

Step 2: Grant macOS Permissions

Go to System Settings → Privacy & Security and enable:

✅ Screen Recording (required for screenshots)
✅ Accessibility (required for UI control)
✅ Automation/AppleScript (optional, for certain actions)

Step 3: Enable Peekaboo

Option A: Via OpenClaw Settings

Settings → Skills → Enable Peekaboo
This installs peekaboo via Homebrew automatically

Option B: Manual Install

brew install peekaboo

Step 4: Enable Peekaboo Bridge

In OpenClaw.app Settings, enable Peekaboo Bridge.

This is crucial — it lets peekaboo reuse the OpenClaw app's permission context, avoiding separate permission grants.

Step 5: Configure Exec Approvals

In OpenClaw.app → Exec approvals, allowlist the peekaboo binary. Don't jump straight to "full" unless you're confident about security.

Verify Your Setup

Run these sanity checks in Terminal:

# Check permissions are granted
peekaboo permissions

# Verify bridge is working
peekaboo bridge status --verbose

# Test screenshot with annotations
peekaboo see --annotate --path /tmp/peekaboo-see.png

If all three pass, you're ready for UI automation!

Using Peekaboo for Automation

Once set up, your agent can:

# Click at coordinates or on elements
peekaboo click --x 100 --y 200

# Type text
peekaboo type "Hello world"

# Window management
peekaboo window list
peekaboo window focus --app "Safari"

# Menu navigation
peekaboo menu click --app "Finder" --menu "File" --item "New Window"

Remote Mac Mini Setup

If your Gateway runs elsewhere and your Mac mini is a remote node:

Pair the Mac mini as a node
Use nodes.run to execute peekaboo commands remotely
Screen captures flow back via the node protocol

Resources

Thanks to the Discord community for the questions that prompted this guide!