react-native-agentkit

Convert React Native app flows to CLI for AI agent automation.

What it does

react-native-agentkit lets AI agents and CLI tools control your React Native app programmatically. Wrap your app with <AgentKitProvider>, and the library automatically discovers all interactive elements (buttons, inputs, scrollviews, switches) and exposes them through a command interface.

Installation

npm install react-native-agentkit
# or
yarn add react-native-agentkit

Quick Start

1. Wrap your app

import { AgentKitProvider } from 'react-native-agentkit';

export default function App() {
  return (
    <AgentKitProvider debug={__DEV__} relayUrl="ws://localhost:8347">
      <YourApp />
    </AgentKitProvider>
  );
}

That's it! The library automatically:

• Scans your component tree for interactive elements
• Starts a command bridge on the device
• Accepts commands from CLI tools

2. Start the relay server

The CLI communicates with the app through a lightweight WebSocket relay:

npx react-native-agentkit-relay
# Listening on port 8347

3. Control via CLI

# Interactive REPL
npx react-native-agentkit connect --relay=ws://localhost:8347

# Execute single commands
npx react-native-agentkit exec list --relay=ws://localhost:8347
npx react-native-agentkit exec tap submit-btn --relay=ws://localhost:8347
npx react-native-agentkit exec type email-input "user@test.com" --relay=ws://localhost:8347

# Pipe mode for AI agents
echo '{"cmd":"list"}' | npx react-native-agentkit pipe --relay=ws://localhost:8347

Commands

Response Format

All commands return structured JSON:

{
  "success": true,
  "command": "tap",
  "target": "submit-btn",
  "result": { "elementTapped": true },
  "screenState": { "elements": [...], "count": 5 },
  "timestamp": 1710000000000
}

Props

Production Usage (Cloud Relay)

In production builds, Metro isn't available. The cloud relay enables AI agents to control the app over the network.

Architecture

┌──────────┐        ┌───────────────┐        ┌──────────────┐
│  CLI /   │◄──WS──►│  Relay Server │◄──WS──►│   RN App     │
│ AI Agent │        │  (Node.js)    │        │ (production) │
└──────────┘        └───────────────┘        └──────────────┘

1. Start the relay server

# Default port 8347
npx react-native-agentkit-relay

# Custom port
npx react-native-agentkit-relay --port=9000

2. Configure the app

<AgentKitProvider
  relayUrl="ws://your-relay-host:8347"
  channelId="my-app"
  channelSecret="your-shared-secret"
>
  <YourApp />
</AgentKitProvider>

3. Connect via CLI

# Interactive REPL
npx react-native-agentkit connect --relay=ws://your-relay-host:8347 --channel=my-app --secret=your-shared-secret

# Execute a command
npx react-native-agentkit exec list --relay=ws://your-relay-host:8347 --channel=my-app --secret=your-shared-secret

# Pipe mode (for AI agents)
echo '{"cmd":"list"}' | npx react-native-agentkit pipe --relay=ws://your-relay-host:8347 --secret=your-shared-secret

Authentication

The relay supports shared-secret authentication to prevent unauthorized access:

• Dev mode (default): Auth is optional. If no secret is set, anyone can connect to the channel.
• Production: Set channelSecret in the app and --secret in the CLI. The first client to join a channel with a secret "locks" that channel — subsequent clients must present matching secrets.
• Enforced auth: Start the relay with --require-auth to reject all unauthenticated connections.

# Start relay with enforced auth
npx react-native-agentkit-relay --require-auth

💡 Tip: Use a unique channelId + channelSecret per app/device for isolation and security.

Hooks

import {
  useAgentKit, // Full context access
  useAgentKitElement, // Manually register an element
  useAgentKitRescan, // Trigger a re-scan
  useAgentKitStatus, // Monitor bridge status
} from 'react-native-agentkit';

Manual Element Registration

For custom components that aren't auto-discovered:

function CustomButton({ onPress, label }) {
  const ref = useRef(null);

  useAgentKitElement(
    'my-btn',
    {
      type: 'button',
      label,
      state: { disabled: false, selected: false },
      position: { x: 0, y: 0, width: 0, height: 0 },
      children: [],
      actions: ['tap'],
    },
    ref,
    { onPress }
  );

  return (
    <Pressable ref={ref} onPress={onPress}>
      <Text>{label}</Text>
    </Pressable>
  );
}

Element Discovery

Elements are discovered via:

1. testID prop — Always included, highest priority
2. Accessibility props — accessibilityLabel, accessibilityRole
3. Interactive handlers — onPress, onChangeText, onValueChange
4. Component type — Pressable, TextInput, Switch, ScrollView, etc.

💡 Tip: Use testID on key elements for reliable AI agent targeting.

Native Dialog Handling

AgentKit automatically intercepts native Alert.alert(), PermissionsAndroid, and react-native-permissions dialogs:

• Conditional suppression — Native dialogs are only suppressed when an agent is connected. When no agent is connected, they display normally.
• Alert buttons: alert-{title}-{button-text}
• Permission buttons: permission-{name}-grant / permission-{name}-deny / permission-{name}-block
• Buttons auto-clean up after being tapped or after a timeout

# After an action triggers Alert.alert('Confirm', '...', [{text:'Cancel'}, {text:'Delete'}])
list                              # Shows: alert-confirm-cancel, alert-confirm-delete
tap alert-confirm-delete          # Executes the Delete button's onPress handler

# Permission request (via PermissionsAndroid or react-native-permissions)
list                              # Shows: permission-camera-grant, permission-camera-deny, permission-camera-block
tap permission-camera-grant       # Grants the permission

💡 Tip: Apps using react-native-permissions get automatic interception for both iOS and Android.

Device Management

AgentKit includes OS-level device management so AI agents can autonomously boot simulators, install apps, grant permissions, and launch — without any manual setup.

💡 Note: Device commands are optional. If the app is already running and connected to the relay, the agent skips device setup and goes straight to UI commands.

CLI Commands

npx react-native-agentkit device list [--platform=ios|android]
npx react-native-agentkit device boot [--name="iPhone 16"]
npx react-native-agentkit device shutdown <udid>
npx react-native-agentkit device install <path> [--device=<udid>] [--expo]
npx react-native-agentkit device uninstall <bundle-id> [--device=<udid>]
npx react-native-agentkit device launch <bundle-id> [--device=<udid>] [--expo]
npx react-native-agentkit device terminate <bundle-id> [--device=<udid>]
npx react-native-agentkit device erase <udid>
npx react-native-agentkit device create-emulator <name> [--image=<system-image>]
npx react-native-agentkit device grant-permissions --bundleId=<id> --permissions=camera,photos,...
npx react-native-agentkit device reset-permissions --bundleId=<id>
npx react-native-agentkit device clear-data <package-id> [--device=<serial>]
npx react-native-agentkit device status

All commands output JSON for pipe-mode compatibility.

Permission Pre-Granting

Grant OS-level permissions before app launch to prevent native permission dialogs during automation:

# iOS (via xcrun simctl privacy)
npx react-native-agentkit device grant-permissions \
  --bundleId=com.myapp \
  --permissions=camera,photos,location,notifications

# Android (via adb shell pm grant)
npx react-native-agentkit device grant-permissions \
  --packageId=com.myapp \
  --permissions=CAMERA,ACCESS_FINE_LOCATION

On Android, device install grants all manifest-declared permissions by default.

Physical iOS Devices

For physical device deployment, AgentKit uses ios-deploy (auto-installed via Homebrew if missing):

npx react-native-agentkit device list --platform=ios  # Shows simulators + physical devices
npx react-native-agentkit device install ./MyApp.app --device=<udid>
npx react-native-agentkit device launch com.myapp --device=<udid>

Expo Integration

For Expo-managed projects, use the --expo flag:

npx react-native-agentkit device install --expo --platform=ios
npx react-native-agentkit device launch --expo --platform=android

AI Agent Prompts

Copy-paste this prompt to give an AI agent (Claude Code, Cursor, etc.) full control of your app — including device setup, permissions, and UI automation:

You can control a React Native app using `react-native-agentkit`.

Read `node_modules/react-native-agentkit/SKILL.md` for the full command reference,
device management, and workflow patterns. It covers everything: booting simulators,
installing apps, granting permissions, and interacting with the UI.

Relay: ws://localhost:8347

Now please: [describe what you want the agent to do]

For production relays with authentication, add connection details:

Relay: ws://your-relay-host:8347 --channel=your-channel --secret=your-secret

Tip: The agent will read SKILL.md and autonomously handle everything — checking if the app is running, booting a device if needed, pre-granting permissions, and then interacting with the UI.

Running the Example

# 1. Install dependencies
yarn install

# 2. Start the relay server (in a separate terminal)
npx react-native-agentkit-relay

# 3. Start the example app on iOS simulator
cd example
npx expo start --ios

# 4. Connect the CLI (in a separate terminal)
npx react-native-agentkit connect --relay=ws://localhost:8347

# Or execute a single command
npx react-native-agentkit exec list --relay=ws://localhost:8347

License

MIT