@convai/web-sdk

The convai/web-sdk helps power characters in websites (brain) and the communication UI with Convai characters. This SDK facilitates the capture of user audio streams and provides appropriate responses in the form of audio, actions, and facial expressions.

Installation

npm install @convai/web-sdk

Quick Start

import { useConvaiClient, ConvaiWidget } from "@convai/web-sdk";

function App() {
  const convaiClient = useConvaiClient({
    apiKey: "your-convai-api-key",
    characterId: "your-character-id",
    enableVideo: true,
    startWithVideoOn: false,
  });

  return (
    <ConvaiWidget
      convaiClient={convaiClient}
      showVideo={true}
      showScreenShare={true}
    />
  );
}

Get Convai Credentials

Visit convai.com and create an account
Navigate to your dashboard
Create a new character or use an existing one
Copy your API Key from the dashboard
Copy your Character ID from the character details

API Reference

useConvaiClient()

Main hook for managing Convai connections. Pass configuration to store it for later use when connecting.

const convaiClient = useConvaiClient({
  apiKey: 'your-api-key',
  characterId: 'your-character-id',
  enableVideo: true,
  startWithVideoOn: false,
});

Note: The hook stores the configuration but does not auto-connect. Connection happens when convaiClient.connect() is called (automatically by ConvaiWidget on first click, or manually in custom implementations).

Returns:

{
  state: ConvaiClientState;           // Connection and activity state
  connectionType: 'audio' | 'video' | null; // Current connection type
  apiKey: string | null;              // Stored API key
  characterId: string | null;         // Stored character ID
  connect: (config?) => Promise<void>; // Connect using stored or override config
  disconnect: () => Promise<void>;    // Disconnect from Convai
  reconnect: () => Promise<void>;     // Disconnect and reconnect
  resetSession: () => void;           // Reset the session ID for fresh start
  room: Room;                         // Internal LiveKit room instance
  sendUserTextMessage: (text) => void;       // Send text message
  sendTriggerMessage: (name?, msg?) => void; // Send trigger message
  updateTemplateKeys: (keys) => void;        // Update template keys
  updateDynamicInfo: (info) => void;         // Update dynamic info
  chatMessages: ChatMessage[];        // Message history
  audioControls: AudioControls;       // Audio control methods
  videoControls: VideoControls;       // Video control methods
  screenShareControls: ScreenShareControls; // Screen share controls
  toggleTts: (enabled) => void;       // Toggle text-to-speech
}

State Object:

interface ConvaiClientState {
  isConnected: boolean; // Connected to Convai
  isConnecting: boolean; // Connection in progress
  isListening: boolean; // Listening to user
  isThinking: boolean; // Processing response
  isSpeaking: boolean; // Speaking to user
  agentState:
    | "disconnected"
    | "connected"
    | "listening"
    | "thinking"
    | "speaking";
}

Audio Controls:

audioControls: {
  isAudioMuted: boolean;
  toggleAudio: () => Promise<void>;
  muteAudio: () => Promise<void>;
  unmuteAudio: () => Promise<void>;
}

Video Controls:

videoControls: {
  isVideoEnabled: boolean;
  enableVideo: () => Promise<void>;
  disableVideo: () => Promise<void>;
}

Screen Share Controls:

screenShareControls: {
  isScreenShareActive: boolean;
  toggleScreenShare: () => Promise<void>;
}

ConvaiWidget Component

Complete all-in-one chat widget with automatic connection, voice mode, and video support.

Features:

Collapsed circular button that expands on click
Connects on first user click (using config from useConvaiClient)
Text and voice mode support
Video controls with floating video window (when connection type is video)
Screen sharing capability (when connection type is video)

<ConvaiWidget
  convaiClient={convaiClient}
  showVideo={true}
  showScreenShare={true}
/>

Props:

Prop	Type	Required	Default	Description
`convaiClient`	`ConvaiClient`	Yes	-	ConvaiClient instance from useConvaiClient() with stored config
`showVideo`	`boolean`	No	`true`	Show video toggle button (only works if connection type is video)
`showScreenShare`	`boolean`	No	`true`	Show screen share button (only works if connection type is video)

Configuration

Config Interface:

interface ConvaiConfig {
  apiKey: string; // Required: Your Convai API key
  characterId: string; // Required: Character ID
  url?: string; // Optional: Custom API URL
  enableVideo?: boolean; // Optional: Enable video capability (default: false)
                         // If true, connection_type is "video" (audio + video + screenshare)
                         // If false, connection_type is "audio" (audio only)
  startWithVideoOn?: boolean; // Optional: Start with camera on (default: false)
                              // Only works if enableVideo is true
  ttsEnabled?: boolean; // Optional: Enable text-to-speech (default: true)
  actionConfig?: ActionConfig; // Optional: Action configuration
}

Note: Audio is always enabled - the microphone is always active when connected. Use enableVideo to control whether video capabilities are available.

Advanced Usage

Sending Messages

// Send text message
convaiClient.sendUserTextMessage("Hello!");

// Send trigger message
convaiClient.sendTriggerMessage("greet", "Custom greeting");

// Update template keys
convaiClient.updateTemplateKeys({
  user_name: "John",
  location: "New York",
});

// Update dynamic info
convaiClient.updateDynamicInfo({
  text: "Additional context for the character",
});

Control Audio/Video

// Toggle microphone
await convaiClient.audioControls.toggleAudio();

// Mute/unmute
await convaiClient.audioControls.muteAudio();
await convaiClient.audioControls.unmuteAudio();

// Enable/disable video
await convaiClient.videoControls.enableVideo();
await convaiClient.videoControls.disableVideo();

// Toggle screen share
await convaiClient.screenShareControls.toggleScreenShare();

Session Management

// Reset session (clears history)
convaiClient.resetSession();

// Disconnect
await convaiClient.disconnect();

// Reconnect with new session
await convaiClient.connect(config);

Advanced Usage

Custom Audio/Video Rendering

For custom implementations, you can use the exported AudioRenderer and AudioContext components:

import { useConvaiClient, AudioRenderer, AudioContext } from "@convai/web-sdk";

function CustomApp() {
  const convaiClient = useConvaiClient();

  return (
    <AudioContext.Provider value={convaiClient.room}>
      {/* AudioRenderer is required for audio playback */}
      <AudioRenderer />
      
      {/* Your custom UI here */}
      <YourCustomUI />
    </AudioContext.Provider>
  );
}

Note: AudioRenderer and AudioContext are re-exported from @livekit/components-react for convenience. These are required for proper audio/video functionality when building custom UIs instead of using ConvaiWidget.

TypeScript Support

The SDK is written in TypeScript and provides full type definitions. Import types as needed:

import {
  ConvaiClient,
  ConvaiConfig,
  ConvaiClientState,
  ChatMessage,
  AudioRenderer,
  AudioContext,
} from "@convai/web-sdk";

License

MIT