@convai/web-sdk

The convai/web-sdk helps power characters in websites (brain) and the communication UI with Convai characters. This SDK facilitates the capture of user audio streams and provides appropriate responses in the form of audio, actions, and facial expressions.

Installation

npm install @convai/web-sdk

Quick Start

import { useConvaiClient, ConvaiWidget } from "@convai/web-sdk";

function App() {
  const convaiClient = useConvaiClient();

  return (
    <ConvaiWidget
      convaiClient={convaiClient}
      apiKey="your-convai-api-key"
      characterId="your-character-id"
      enableVideo={true}
      startWithVideoOn={false}
      enableAudio={true}
    />
  );
}

Get Convai Credentials

Visit convai.com and create an account
Navigate to your dashboard
Create a new character or use an existing one
Copy your API Key from the dashboard
Copy your Character ID from the character details

API Reference

useConvaiClient()

Main hook for managing Convai connections.

const convaiClient = useConvaiClient();

Returns:

{
  state: ConvaiClientState;           // Connection and activity state
  connect: (config) => Promise<void>; // Connect to Convai
  disconnect: () => Promise<void>;    // Disconnect from Convai
  resetSession: () => void;           // Reset the session ID
  room: Room;                         // Internal room instance
  sendUserTextMessage: (text) => void;       // Send text message
  sendTriggerMessage: (name?, msg?) => void; // Send trigger message
  updateTemplateKeys: (keys) => void;        // Update template keys
  updateDynamicInfo: (info) => void;         // Update dynamic info
  chatMessages: ChatMessage[];        // Message history
  audioControls: AudioControls;       // Audio control methods
  videoControls: VideoControls;       // Video control methods
  screenShareControls: ScreenShareControls; // Screen share controls
  toggleTts: (enabled) => void;       // Toggle text-to-speech
}

State Object:

interface ConvaiClientState {
  isConnected: boolean; // Connected to Convai
  isConnecting: boolean; // Connection in progress
  isListening: boolean; // Listening to user
  isThinking: boolean; // Processing response
  isSpeaking: boolean; // Speaking to user
  agentState:
    | "disconnected"
    | "connected"
    | "listening"
    | "thinking"
    | "speaking";
}

Audio Controls:

audioControls: {
  isAudioMuted: boolean;
  toggleAudio: () => Promise<void>;
  muteAudio: () => Promise<void>;
  unmuteAudio: () => Promise<void>;
}

Video Controls:

videoControls: {
  isVideoEnabled: boolean;
  enableVideo: () => Promise<void>;
  disableVideo: () => Promise<void>;
}

Screen Share Controls:

screenShareControls: {
  isScreenShareActive: boolean;
  toggleScreenShare: () => Promise<void>;
}

ConvaiWidget Component

Complete all-in-one chat widget with automatic connection, voice mode, and video support.

Features:

Collapsed circular button that expands on click
Auto-connects on first interaction
Text and voice mode support
Video controls with floating video window
Screen sharing capability

<ConvaiWidget
  convaiClient={convaiClient}
  apiKey="your-api-key"
  characterId="your-character-id"
  enableVideo={true}
  startWithVideoOn={false}
  enableAudio={true}
  url="https://your-custom-url.com"
/>

Props:

Prop	Type	Required	Default	Description
`convaiClient`	`ConvaiClient`	Yes	-	ConvaiClient instance from useConvaiClient()
`apiKey`	`string`	Yes	-	Your Convai API key
`characterId`	`string`	No	-	Character ID to connect to
`enableVideo`	`boolean`	No	`false`	Enable video capability (shows video toggle button)
`startWithVideoOn`	`boolean`	No	`false`	Start with camera on when connecting
`enableAudio`	`boolean`	No	`true`	Enable audio
`url`	`string`	No	-	Custom Convai API URL

Configuration

Config Interface:

interface ConvaiConfig {
  apiKey: string; // Required: Your Convai API key
  characterId: string; // Required: Character ID
  url?: string; // Optional: Custom API URL
  enableVideo?: boolean; // Optional: Enable video (default: false)
  enableAudio?: boolean; // Optional: Enable audio (default: true)
  actionConfig?: ActionConfig; // Optional: Action configuration
}

Advanced Usage

Sending Messages

// Send text message
convaiClient.sendUserTextMessage("Hello!");

// Send trigger message
convaiClient.sendTriggerMessage("greet", "Custom greeting");

// Update template keys
convaiClient.updateTemplateKeys({
  user_name: "John",
  location: "New York",
});

// Update dynamic info
convaiClient.updateDynamicInfo({
  text: "Additional context for the character",
});

Control Audio/Video

// Toggle microphone
await convaiClient.audioControls.toggleAudio();

// Mute/unmute
await convaiClient.audioControls.muteAudio();
await convaiClient.audioControls.unmuteAudio();

// Enable/disable video
await convaiClient.videoControls.enableVideo();
await convaiClient.videoControls.disableVideo();

// Toggle screen share
await convaiClient.screenShareControls.toggleScreenShare();

Session Management

// Reset session (clears history)
convaiClient.resetSession();

// Disconnect
await convaiClient.disconnect();

// Reconnect with new session
await convaiClient.connect(config);

Advanced Usage

Custom Audio/Video Rendering

For custom implementations, you can use the exported AudioRenderer and AudioContext components:

import { useConvaiClient, AudioRenderer, AudioContext } from "@convai/web-sdk";

function CustomApp() {
  const convaiClient = useConvaiClient();

  return (
    <AudioContext.Provider value={convaiClient.room}>
      {/* AudioRenderer is required for audio playback */}
      <AudioRenderer />
      
      {/* Your custom UI here */}
      <YourCustomUI />
    </AudioContext.Provider>
  );
}

Note: AudioRenderer and AudioContext are re-exported from @livekit/components-react for convenience. These are required for proper audio/video functionality when building custom UIs instead of using ConvaiWidget.

TypeScript Support

The SDK is written in TypeScript and provides full type definitions. Import types as needed:

import {
  ConvaiClient,
  ConvaiConfig,
  ConvaiClientState,
  ChatMessage,
  AudioRenderer,
  AudioContext,
} from "@convai/web-sdk";

License

MIT