Package Exports
- @loonylabs/ttv-middleware
- @loonylabs/ttv-middleware/dist/index.js
This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package (@loonylabs/ttv-middleware) to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.
Readme
TTV Middleware
Provider-agnostic Text-to-Video middleware with async polling, retry logic, and comprehensive error handling. Currently supports OpenAI Sora (Sora 2, Sora 2 Pro) and Google Veo (Veo 2, Veo 3, Veo 3.1). Features image-to-video, video extension, configurable progress callbacks, and optional auto-download.
Table of Contents
Features
- Multi-Provider Architecture: Unified API for all TTV providers
- OpenAI Sora: Sora 2 & Sora 2 Pro with text-to-video and image-to-video
- Google Veo: Veo 2, Veo 3, Veo 3.1 (+ fast variants) with video extension support
- Async Polling: Both APIs are asynchronous - the middleware handles polling internally with configurable intervals and exponential backoff
- Image-to-Video: Animate a still image into a video (both providers)
- Video Extension: Continue/extend an existing video (Google Veo)
- Progress Callbacks: Optional
onProgresscallback for real-time generation status updates - Auto-Download: Optionally download generated videos to Buffer (
downloadToBuffer) - Retry Logic: Exponential backoff with jitter for transient errors (429, 408, 5xx, timeouts)
- TypeScript First: Full type safety with comprehensive interfaces
- Logging Control: Configurable log levels via environment or API
- Debug Logging: Markdown file logging for debugging prompts and responses
- Error Handling: Typed error classes including
PollingTimeoutErrorandContentModeratedError - Dry Mode: Validate requests without API calls (no costs during development)
Quick Start
Installation
Install from npm:
npm install @loonylabs/ttv-middleware
# For OpenAI Sora provider:
npm install openai
# For Google Veo provider:
npm install google-auth-libraryOr install directly from GitHub:
npm install github:loonylabs-dev/ttv-middlewareBasic Usage
import { TTVService, OpenAISoraProvider, TTVProvider } from '@loonylabs/ttv-middleware';
// Create service and register provider
const service = new TTVService();
service.registerProvider(new OpenAISoraProvider({
apiKey: process.env.OPENAI_API_KEY,
}));
// Generate a video
const result = await service.generate({
prompt: 'A cat sitting on a windowsill watching rain fall outside',
model: 'sora-2',
duration: 8, // 8 seconds
aspectRatio: '16:9',
});
console.log('Video URL:', result.videos[0].url);
console.log('Duration:', result.metadata.duration, 'ms (total)');
console.log('Generation time:', result.metadata.generationTime, 'ms');Using Google Veo
import { TTVService, GoogleVeoProvider, TTVProvider } from '@loonylabs/ttv-middleware';
const service = new TTVService();
service.registerProvider(new GoogleVeoProvider({
projectId: process.env.GOOGLE_CLOUD_PROJECT,
location: 'us-central1',
}));
const result = await service.generate({
prompt: 'A cinematic aerial shot of a mountain range at sunrise',
model: 'veo-3.0-generate-001',
duration: 8,
aspectRatio: '16:9',
resolution: '1080p',
generateAudio: true, // Veo 3+ generates audio natively
});
console.log('Video URL:', result.videos[0].url);With Progress Callback
const result = await service.generate({
prompt: 'A futuristic city with flying cars',
model: 'sora-2',
duration: 12,
onProgress: (progress) => {
console.log(`Status: ${progress.status}`, progress.message || '');
// Status: queued
// Status: in_progress
// Status: completed
},
});Download to Buffer
import * as fs from 'fs';
const result = await service.generate({
prompt: 'Ocean waves crashing on a rocky shore',
model: 'sora-2',
downloadToBuffer: true, // Download video into memory
});
// Save to disk
fs.writeFileSync('output.mp4', result.videos[0].buffer!);Switching Providers
// Use OpenAI Sora
const soraResult = await service.generate({
prompt: 'A mountain landscape timelapse',
model: 'sora-2',
}, TTVProvider.OPENAI_SORA);
// Use Google Veo
const veoResult = await service.generate({
prompt: 'A mountain landscape timelapse',
model: 'veo-3.0-generate-001',
}, TTVProvider.GOOGLE_VEO);Prerequisites
Required Dependencies
- Node.js 18+
- TypeScript 5.3+
For OpenAI Sora provider:
npm install openaiFor Google Veo provider:
npm install google-auth-libraryConfiguration
Environment Setup
Create a .env file in your project root:
# Default provider
TTV_DEFAULT_PROVIDER=openai-sora
# Logging level (debug, info, warn, error, silent)
TTV_LOG_LEVEL=info
# OpenAI Sora
OPENAI_API_KEY=sk-...
# Google Veo (Vertex AI)
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_APPLICATION_CREDENTIALS=./service-account.json
GOOGLE_CLOUD_REGION=us-central1Providers & Models
OpenAI Sora
| Model | ID | Duration | Resolutions | Audio | Image-to-Video |
|---|---|---|---|---|---|
| Sora 2 | sora-2 |
4, 8, 12s | 720p, 1080p | Yes | Yes |
| Sora 2 Pro | sora-2-pro |
10, 15, 25s | 720p, 1080p | Yes | Yes |
Pricing: ~$0.10-$0.50/sec depending on model and resolution.
Google Veo
| Model | ID | Duration | Resolutions | Audio | Video Extension |
|---|---|---|---|---|---|
| Veo 2 | veo-2.0-generate-001 |
5-8s | 720p | No | Yes |
| Veo 3 | veo-3.0-generate-001 |
4-8s | 720p, 1080p | Yes | Yes |
| Veo 3 Fast | veo-3.0-fast-generate-001 |
4-8s | 720p, 1080p | Yes | Yes |
| Veo 3.1 | veo-3.1-generate-001 |
4-8s | 720p, 1080p, 4K | Yes | Yes |
| Veo 3.1 Fast | veo-3.1-fast-generate-001 |
4-8s | 720p, 1080p, 4K | Yes | Yes |
Pricing: ~$0.15-$0.75/sec depending on model.
Image-to-Video
Animate a still image into a video. Both providers support this:
import * as fs from 'fs';
// Load a reference image
const imageBase64 = fs.readFileSync('character.png').toString('base64');
// OpenAI Sora: image becomes first frame
const result = await service.generate({
prompt: 'She turns around and smiles, then slowly walks out of frame',
model: 'sora-2',
referenceImage: {
base64: imageBase64,
mimeType: 'image/png',
},
duration: 8,
});
// Google Veo: first frame + optional last frame for interpolation
const veoResult = await service.generate({
prompt: 'Camera slowly zooms out revealing the landscape',
model: 'veo-3.0-generate-001',
referenceImage: {
base64: firstFrameBase64,
mimeType: 'image/png',
},
lastFrameImage: { // Veo-specific: interpolate between two keyframes
base64: lastFrameBase64,
mimeType: 'image/png',
},
}, TTVProvider.GOOGLE_VEO);Video Extension
Extend an existing video (Google Veo only):
const extended = await service.extend({
prompt: 'The camera continues to pan revealing a hidden waterfall',
videoBuffer: existingVideoBuffer,
videoMimeType: 'video/mp4',
duration: 7, // Extend by 7 seconds
downloadToBuffer: true,
}, TTVProvider.GOOGLE_VEO);
fs.writeFileSync('extended.mp4', extended.videos[0].buffer!);API Reference
TTVService
class TTVService {
registerProvider(provider: BaseTTVProvider): void;
generate(request: TTVRequest, provider?: TTVProvider): Promise<TTVResponse>;
extend(request: TTVExtendRequest, provider?: TTVProvider): Promise<TTVResponse>;
getProvider(name: TTVProvider): BaseTTVProvider | undefined;
getAvailableProviders(): TTVProvider[];
listAllModels(): Array<{ provider: TTVProvider; models: ModelInfo[] }>;
findProvidersWithCapability(capability: keyof TTVCapabilities): Array<{ provider: TTVProvider; models: ModelInfo[] }>;
}TTVRequest
interface TTVRequest {
prompt: string;
model?: string; // 'sora-2', 'veo-3.0-generate-001', etc.
duration?: number; // Desired duration in seconds
aspectRatio?: string; // '16:9', '9:16'
resolution?: '720p' | '1080p' | '4k';
n?: number; // Number of videos (default: 1)
// Image-to-video
referenceImage?: TTVReferenceImage;
lastFrameImage?: TTVReferenceImage; // Google Veo only
// Audio & content
generateAudio?: boolean;
negativePrompt?: string;
// Output control
downloadToBuffer?: boolean; // Download video to Buffer (default: false)
onProgress?: TTVProgressCallback;
// Retry & debug
retry?: boolean | RetryOptions;
dry?: boolean;
providerOptions?: Record<string, unknown>;
}TTVResponse
interface TTVResponse {
videos: TTVVideo[];
metadata: {
provider: string;
model: string;
region?: string;
duration: number; // Total request time (ms)
generationTime?: number; // Polling time only (ms)
};
usage: {
videosGenerated: number;
totalDurationSeconds: number;
modelId: string;
};
billing?: {
cost: number;
currency: string;
source: 'provider' | 'estimated';
};
}TTVVideo
interface TTVVideo {
url?: string; // Video URL (Sora) or undefined (Veo returns buffer directly)
buffer?: Buffer; // Video data (if downloadToBuffer)
contentType: string; // 'video/mp4'
duration?: number; // Duration in seconds
}Advanced Features
Polling Configuration
Video generation is asynchronous. The middleware polls for completion automatically. You can configure the polling behavior:
import { GoogleVeoProvider } from '@loonylabs/ttv-middleware';
const provider = new GoogleVeoProvider({
projectId: 'my-project',
polling: {
intervalMs: 10000, // Start polling every 10s (default)
maxIntervalMs: 30000, // Cap at 30s between polls
backoffMultiplier: 1.5, // Increase interval by 1.5x each time
timeoutMs: 600000, // Give up after 10 minutes (default)
},
});| Option | Default | Description |
|---|---|---|
intervalMs |
10000 | Initial polling interval (ms) |
maxIntervalMs |
30000 | Maximum polling interval (ms) |
backoffMultiplier |
1.5 | Multiplier per poll attempt |
timeoutMs |
600000 | Maximum wait time (10 minutes) |
Retry Configuration
Automatic retry with exponential backoff and jitter for transient errors (429, 408, 5xx, network timeouts):
// Default: 3 retries, exponential backoff (1s -> 2s -> 4s), jitter enabled
const result = await service.generate({
prompt: 'A sunset over mountains',
// retry: true (default)
});
// Custom retry configuration
const result = await service.generate({
prompt: 'A sunset over mountains',
retry: {
maxRetries: 5,
delayMs: 1000,
backoffMultiplier: 2.0,
maxDelayMs: 30000,
jitter: true,
},
});
// Disable retry
const result = await service.generate({
prompt: 'A sunset over mountains',
retry: false,
});Retryable errors: 429, 408, 500, 502, 503, 504, timeouts, ECONNRESET, ECONNREFUSED, socket hang up Not retried: 400, 401, 403, and other client errors
Logging Configuration
Control logging via environment variable or API:
import { setLogLevel } from '@loonylabs/ttv-middleware';
// Set log level programmatically
setLogLevel('warn'); // Only show warnings and errors
// Or via environment variable
// TTV_LOG_LEVEL=errorAvailable levels: debug, info, warn, error, silent
Debug Logging (Markdown Files)
Log all TTV requests and responses to markdown files for debugging:
import { TTVDebugger } from '@loonylabs/ttv-middleware';
// Enable via environment variable
// DEBUG_TTV_REQUESTS=true
// Or programmatically
TTVDebugger.setEnabled(true);
TTVDebugger.setLogsDir('./logs/ttv/requests');
TTVDebugger.configure({
enabled: true,
logsDir: './logs/ttv/requests',
consoleLog: true,
});Error Handling
Typed error classes for precise error handling:
import {
TTVError,
InvalidConfigError,
QuotaExceededError,
ProviderUnavailableError,
GenerationFailedError,
NetworkError,
CapabilityNotSupportedError,
PollingTimeoutError,
ContentModeratedError,
} from '@loonylabs/ttv-middleware';
try {
const result = await service.generate({ prompt: 'test', duration: 8 });
} catch (error) {
if (error instanceof PollingTimeoutError) {
console.log('Video generation timed out - try again or increase timeout');
} else if (error instanceof ContentModeratedError) {
console.log('Content was blocked by safety filters');
} else if (error instanceof QuotaExceededError) {
console.log('Rate limit hit, try again later');
} else if (error instanceof CapabilityNotSupportedError) {
console.log('Model does not support this feature');
} else if (error instanceof TTVError) {
console.log(`TTV Error [${error.code}]: ${error.message}`);
}
}Dry Mode
Test your integration without making API calls or incurring costs:
const result = await service.generate({
prompt: 'A test video',
duration: 8,
dry: true, // No API call, returns placeholder response
});
console.log(result.videos.length); // 1
console.log(result.metadata.duration); // 0 (no actual generation)Provider-Specific Options
Use providerOptions as an escape hatch for provider-specific features:
// Google Veo: seed for deterministic output
const result = await service.generate({
prompt: 'A sunset timelapse',
model: 'veo-3.0-generate-001',
providerOptions: {
seed: 42,
personGeneration: 'allow_adult',
enhancePrompt: true,
storageUri: 'gs://my-bucket/output/', // Direct output to GCS
},
}, TTVProvider.GOOGLE_VEO);
// OpenAI Sora: remix (reinterpret an existing video)
// Use the Sora API directly for remix via providerOptionsTesting
# Run all tests
npm test
# Unit tests only
npm run test:unit
# Unit tests with watch mode
npm run test:unit:watch
# Unit tests with coverage report
npm run test:unit:coverage
# Integration tests (requires TTV_INTEGRATION_TESTS=true)
npm run test:integration
# CI/CD mode
npm run test:ciManual Smoke Tests
The scripts/ directory contains manual smoke tests for real API calls:
# Text-to-video with Google Veo (veo-3.0-fast)
npx ts-node scripts/manual-test-veo.ts
# Image-to-video with Google Veo (veo-3.0-fast)
npx ts-node scripts/manual-test-veo-i2v.tsBoth scripts use veo-3.0-fast-generate-001 (cheapest model), generate short 4s/720p videos, and save output to the output/ directory. Requires Google Veo credentials in .env.
Integration Tests
Integration tests make real API calls and cost money. They are skipped by default.
# Enable and run integration tests
TTV_INTEGRATION_TESTS=true npm run test:integrationPrerequisites:
OPENAI_API_KEYfor Sora testsGOOGLE_CLOUD_PROJECTandGOOGLE_APPLICATION_CREDENTIALSfor Veo tests
Contributing
We welcome contributions! Please ensure:
Tests: Add tests for new features
Linting: Run
npm run lintbefore committingConventions: Follow the existing project structure
Fork the repository
Create your feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add some amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.