Package Exports

This package does not declare an exports field, so the exports above have been automatically detected and optimized by JSPM instead. If any package subpath is missing, it is recommended to post an issue to the original package () to support the "exports" field. If that is not possible, create a JSPM override to customize the exports field for this package.

Readme

Tensorchat Streaming JavaScript Client

Framework-agnostic TypeScript/JavaScript client for Tensorchat.io streaming API. Process multiple LLM prompts concurrently with real-time streaming responses and ultra-fast performance optimizations.

Features

Ultra-Fast Streaming: Zero throttling delays with optimized buffer processing
Framework Agnostic: Works with vanilla JS, React, Vue, Angular, Svelte, or any framework
Real-time UI Updates: Live streaming updates for responsive user interfaces
Concurrent Processing: Handle multiple prompts simultaneously with intelligent buffering
TypeScript Support: Fully typed for better developer experience
Search Integration: Track search progress and completion for enhanced UX
Memory Efficient: Automatic buffer cleanup and minimal memory footprint
Guaranteed Callbacks: Reliable completion callbacks with complete data

Installation

npm install @tensorchat.io/streaming

Quick Start

import { TensorchatStreaming } from "@tensorchat.io/streaming";

const client = new TensorchatStreaming({
  apiKey: "your-api-key",
  baseUrl: "https://api.tensorchat.ai", // optional
  verbose: false // optional, default false
});

const request = {
  context: "Analyze the following data",
  model: "gpt-4",
  tensors: [
    { messages: "Summarize key trends", search: true },
    { messages: "Extract insights", search: false },
    { messages: "Generate recommendations", search: true }
  ]
};

await client.streamProcess(request, {
  onSearchProgress: (data) => {
    console.log(`Searching for tensor ${data.index}...`);
  },
  
  onSearchComplete: (data) => {
    console.log(`Search complete for tensor ${data.index}`);
  },
  
  onTensorChunk: (data) => {
    // Real-time UI updates - called for each chunk
    console.log(`Tensor ${data.index}: ${data.chunk}`);
    updateUI(data.index, data.chunk);
  },
  
  onTensorComplete: (data) => {
    // Final callback with complete data
    console.log(`Tensor ${data.index} complete`);
    console.log(`Content: ${data.result.content}`);
    console.log(`Total chunks: ${data.streamBuffers.length}`);
  },
  
  onComplete: (data) => {
    console.log("All tensors processed");
  },
  
  onError: (error) => {
    console.error("Processing error:", error);
  }
});

// Clean up when done
client.destroy();

Advanced Usage

React Integration

import React, { useState, useEffect, useRef } from 'react';
import { TensorchatStreaming } from '@tensorchat.io/streaming';

function StreamingComponent() {
  const [tensorContents, setTensorContents] = useState({});
  const [searchStatuses, setSearchStatuses] = useState({});
  const [isProcessing, setIsProcessing] = useState(false);
  const clientRef = useRef(null);

  useEffect(() => {
    clientRef.current = new TensorchatStreaming({
      apiKey: process.env.REACT_APP_TENSORCHAT_API_KEY,
      verbose: true
    });

    return () => clientRef.current?.destroy();
  }, []);

  const processData = async () => {
    setIsProcessing(true);
    setTensorContents({});
    setSearchStatuses({});

    const request = {
      context: "Market analysis context",
      model: "gpt-4",
      tensors: [
        { messages: "Analyze crypto trends", search: true },
        { messages: "Stock market overview", search: true },
        { messages: "Economic predictions", search: false }
      ]
    };

    try {
      await clientRef.current.streamProcess(request, {
        onSearchProgress: (data) => {
          setSearchStatuses(prev => ({
            ...prev,
            [data.index]: 'searching'
          }));
        },

        onSearchComplete: (data) => {
          setSearchStatuses(prev => ({
            ...prev,
            [data.index]: 'complete'
          }));
        },

        onTensorChunk: (data) => {
          setTensorContents(prev => ({
            ...prev,
            [data.index]: (prev[data.index] || '') + data.chunk
          }));
        },

        onTensorComplete: (data) => {
          // Final validation and processing
          console.log(`Tensor ${data.index} final content:`, data.result.content);
        },

        onComplete: () => {
          setIsProcessing(false);
        },

        onError: (error) => {
          console.error('Streaming error:', error);
          setIsProcessing(false);
        }
      });
    } catch (error) {
      console.error('Request failed:', error);
      setIsProcessing(false);
    }
  };

  return (
    <div>
      <button onClick={processData} disabled={isProcessing}>
        {isProcessing ? 'Processing...' : 'Start Analysis'}
      </button>

      {Object.entries(tensorContents).map(([index, content]) => (
        <div key={index} style={{ margin: '20px 0', padding: '15px', border: '1px solid #ccc' }}>
          <h3>
            Tensor {index}
            {searchStatuses[index] === 'searching' && ' (Searching...)'}
            {searchStatuses[index] === 'complete' && ' (Search Complete)'}
          </h3>
          <div style={{ whiteSpace: 'pre-wrap' }}>{content}</div>
        </div>
      ))}
    </div>
  );
}

export default StreamingComponent;

Framework Manager (Alternative)

For more complex applications, use the framework manager:

import { createTensorchatStreaming } from '@tensorchat.io/streaming';

const manager = createTensorchatStreaming({
  apiKey: 'your-api-key',
  verbose: true
});

// Use the manager
await manager.streamProcess(request, callbacks);

// Update configuration
manager.updateConfig({ verbose: false });

// Clean up
manager.destroy();

API Reference

TensorchatStreaming Class

Constructor Options

interface TensorchatConfig {
  apiKey: string;           // Your Tensorchat API key (required)
  baseUrl?: string;         // API endpoint (default: 'https://api.tensorchat.ai')
  verbose?: boolean;        // Enable debug logging (default: false)
}

Methods

`streamProcess(request, callbacks)`

Process tensors with real-time streaming.

Parameters:

request: StreamRequest object
callbacks: StreamCallbacks object

Returns: Promise

`destroy()`

Clean up resources and clear buffers.

Stream Request Format

interface StreamRequest {
  context: string;          // Context for all tensors
  model: string;            // LLM model to use
  tensors: TensorConfig[];  // Array of tensor configurations
}

interface TensorConfig {
  messages: string;         // The prompt/message
  concise?: boolean;        // Request concise response
  model?: string;           // Override model for this tensor
  search?: boolean;         // Enable search for this tensor
}

Stream Callbacks

interface StreamCallbacks {
  onSearchProgress?: (data: StreamEventData) => void;     // Search in progress
  onSearchComplete?: (data: StreamEventData) => void;     // Search completed
  onTensorChunk?: (data: StreamEventData) => void;        // Streaming content chunk
  onTensorComplete?: (data: StreamEventData) => void;     // Tensor complete with final data
  onComplete?: (data: StreamEventData) => void;           // All tensors complete
  onError?: (error: Error) => void;                       // Error handling
}

Stream Event Data

interface StreamEventData {
  type: string;             // Event type
  index?: number;           // Tensor index
  chunk?: string;           // Content chunk (for tensor_chunk events)
  result?: {
    content: string;        // Complete content (for tensor_complete events)
    // ... other result properties
  };
  streamBuffers?: string[]; // All chunks for this tensor (tensor_complete only)
  error?: string;           // Error message (for error events)
  totalTensors?: number;    // Total number of tensors (for complete events)
}

Performance Optimizations

The client includes several performance optimizations for maximum throughput:

Zero Throttling: No artificial delays in chunk processing
Optimized Buffer Processing: Uses indexOf with start position for efficient string parsing
Memory Efficient: Automatic buffer cleanup after tensor completion
Minimal String Operations: Single join operation per tensor for final content
Smart Callback Management: Guaranteed callback ordering with onTensorComplete as final call

Error Handling

The client provides comprehensive error handling:

await client.streamProcess(request, {
  onError: (error) => {
    console.error('Streaming error:', error.message);
    // Handle different error types
    if (error.message.includes('HTTP 401')) {
      // Handle authentication error
    } else if (error.message.includes('HTTP 429')) {
      // Handle rate limiting
    }
  }
});

Best Practices

Always call destroy() when done to clean up resources
Use onTensorChunk for real-time UI updates
Use onTensorComplete for final processing and validation
Enable search callbacks when using search functionality
Handle errors gracefully with proper error callbacks
Set verbose: true during development for debugging

Browser Compatibility

Modern browsers with fetch API support
Node.js 14+
TypeScript 4.0+

Links & Resources

NPM Package: https://www.npmjs.com/package/@tensorchat.io/streaming
GitHub Repository: https://github.com/datacorridor/tensorchat-streaming
Tensorchat Platform: https://tensorchat.io
API Documentation: https://tensorchat.io/#api-docs

License

MIT License - see LICENSE file for details.

Support

Issues: GitHub Issues
Email: support@datacorridor.io
Documentation: tensorchat.io/#api-docs

Tensorchat.io is a product of Data Corridor Limited