JSPM

  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 214
  • Score
    100M100P100Q103541F
  • License MIT

Zero-dependency TypeScript middleware for dynamic vector space transformation. Warp embeddings at runtime with WASM-accelerated affine transforms, quantization, online learning, and ColBERT โ€” no retraining needed.

Package Exports

  • warpvector
  • warpvector/langchain
  • warpvector/prisma

Readme

warpvector ๐ŸŒŒ

npm version License: MIT Edge Ready Zero Dependencies Tests

Warp your vector space at runtime โ€” no retraining, no Python, just TypeScript.

warpvector is a lightweight, zero-dependency TypeScript middleware that dynamically transforms vector spaces based on search context and user intent, without retraining AI models or running expensive re-inference.

It sits between your embedding model and vector database, applying fast in-memory affine transformations to bring semantic distances closer to the user's true intent.

๐ŸŽฎ Try the Interactive Playground ยท ๐Ÿ“– ๆ—ฅๆœฌ่ชž็‰ˆ README


โšก Results at a Glance

Metric Before (vanilla search) After (WarpVector) Improvement
Int8 Quantization Fidelity โ€” cosine sim 0.9999 Lossless compression
MLP Inference (WASM) โ€” 1.1โ€“3.8 ยตs/vector Near-zero latency
Int8 Quantization Speed โ€” 322K vecs/sec Real-time capable
Binary Quantization Speed โ€” 1.18M vecs/sec Extreme throughput
Memory Reduction (Int8) 6 KB/vec (1536-dim) 1.5 KB/vec 75% reduction
Memory Reduction (Binary) 6 KB/vec (1536-dim) 192 B/vec 96.9% reduction
Pipeline Latency โ€” 119 ยตs (Intent + Projection) Sub-millisecond
IR Accuracy (NDCG@10) 68.2% (vanilla) 77.0% (Intent Warping) +13.0% improvement
Quantization Recall@10 (Int8) โ€” 86โ€“96% Near-lossless retrieval
๐Ÿ“Š Full Benchmark Results
Adapter Dimensions Avg Latency Accuracy Metric Value
IntentAdapter 128D 21.1 ยตs Identity precision 1.000000
IntentAdapter 768D 603.3 ยตs Identity precision 1.000000
IntentAdapter 1536D 2406.2 ยตs Identity precision 1.000000
ProjectionAdapter 1536 โ†’ 512 807.0 ยตs โ€” โ€”
ProjectionAdapter 768 โ†’ 256 204.0 ยตs โ€” โ€”
QuantizationAdapter 128D (int8) 0.7 ยตs Quantization fidelity 0.999992
QuantizationAdapter 768D (int8) 4.2 ยตs Quantization fidelity 0.999992
QuantizationAdapter 1536D (int8) 4.2 ยตs Quantization fidelity 0.999992
MlpAdapter (WASM) 128 โ†’ 64 2.2 ยตs โ€” โ€”
MlpAdapter (WASM) 768 โ†’ 256 3.8 ยตs โ€” โ€”
MlpAdapter (WASM) 1536 โ†’ 512 โ†’ 128 1.1 ยตs โ€” โ€”
Pipeline 768 โ†’ 256 (Intent+Proj) 119.1 ยตs โ€” โ€”

Benchmarked on Apple M-series, Bun runtime. Run bun run benchmarks/accuracy.ts to reproduce.


๐Ÿ’ก Why WarpVector?

Traditional vector search is static โ€” it depends entirely on pre-generated embedding distances. When you need context-aware tuning, your only options have been metadata filtering or expensive re-inference with instruction-tuned models.

WarpVector changes this. It applies lightweight matrix operations at query time, warping the vector space to match user intent โ€” all without touching the base embedding model.

graph LR
    Input["Search Query"] --> LLM["OpenAI / Cohere / etc."]
    LLM -->|"Base Vector"| WP{"WarpPipeline"}
    
    subgraph WarpVector["In-Memory Transformation (sub-ms)"]
        WP --> Step1["MlpAdapter<br/>Non-linear Transform"]
        Step1 --> Step2["IntentAdapter<br/>Domain Warping"]
        Step2 --> Final["QuantizationAdapter<br/>Int8 Compression"]
    end
    
    Final -->|"Optimized Vector"| DB[("Vector DB<br/>Pinecone / pgvector / etc.")]

๐ŸŽฏ Key Use Cases

Standard embeddings can't distinguish "Apple" (fruit) from "Apple" (company). WarpVector lets you switch intents to instantly warp the vector space toward the right domain.

2. Real-Time Online Learning at the Edge

No need to retrain LLMs. Learn from user clicks and skips directly on Cloudflare Workers or Vercel Edge โ€” updating only the lightweight transformation matrix, not the model itself.

3. Auto-Correction of Embedding Anisotropy

Many embedding models produce vectors that are all too similar (anisotropy). WhiteningAdapter automatically learns and removes this bias via streaming Online PCA, dramatically improving search resolution.

4. 75โ€“97% Memory Reduction via Quantization

Add .setFinalStage("quantize", quantizer) to your pipeline to compress vectors from Float32 to Int8 (4ร— reduction) or Binary (32ร— reduction) with 0.9999+ cosine similarity preservation.

5. Drop-in Integration โ€” Just a Few Lines

No Python. No heavy ML frameworks. Pure TypeScript + WASM. Works with LangChain, Prisma (pgvector), and LlamaIndex out of the box.


๐Ÿ“ฆ Installation

npm install warpvector
# or
bun add warpvector

All core features work with zero dependencies. For integrations:

# Prisma + pgvector
npm install @prisma/client sql-template-tag

# LangChain
npm install @langchain/core

๐Ÿš€ Quick Start

import { WarpPipeline } from 'warpvector';

const pipeline = new WarpPipeline(1536)
  .addIntent({ tech: { matrix: techMatrix, bias: techBias } })
  .setFinalStage("quantize", new QuantizationAdapter({ type: "int8", dim: 1536 }));

// Auto-initializes WASM on first call โ€” no manual init() needed
const result = pipeline.run(baseVector, { intent: "tech" });

Intent-Aware Transformation

import { IntentAdapter } from 'warpvector';

const adapter = new IntentAdapter(1536);
adapter.addIntent("technical", { matrix: techMatrix, bias: techBias });
adapter.addIntent("business",  { matrix: bizMatrix,  bias: bizBias  });

// Same vector, different results based on intent
const techResult = adapter.tune(queryVector, "technical");
const bizResult  = adapter.tune(queryVector, "business");

Auto-Generate Intent Matrices (NEW)

import { IntentMatrixFactory } from 'warpvector/ml';
import { IntentAdapter } from 'warpvector';

// No manual matrix needed โ€” just provide category samples
const factory = new IntentMatrixFactory(1536);
factory.addCategory("tech", [techVec1, techVec2, techVec3]);
factory.addCategory("business", [bizVec1, bizVec2, bizVec3]);

// Auto-learns optimal affine transformations via contrastive learning
const intents = await factory.build();

const adapter = new IntentAdapter(1536);
adapter.addIntent("tech", intents.tech);
adapter.addIntent("business", intents.business);

// Auto-blending: automatically routes queries to the best intent
const result = adapter.tuneAutoBlended(queryVector);

WASM-Accelerated Neural Network Inference

import { MlpAdapter } from 'warpvector/ml';

const mlp = new MlpAdapter([
  { matrix: layer1Weights, bias: layer1Bias, activation: "relu" },
  { matrix: layer2Weights, bias: layer2Bias, activation: "linear" },
]);
await mlp.init(); // Load WASM

const output = mlp.tune(inputVector); // ~2ยตs per inference

Online Whitening (Auto-fix Embedding Anisotropy)

import { WhiteningAdapter } from 'warpvector/ml';

const adapter = new WhiteningAdapter(1536, { learningRate: 0.01, numComponents: 1 });

// Streaming learning โ€” call update() with each incoming vector
adapter.update(vector1);
adapter.update(vector2);

// Apply whitening to remove learned bias
const improved = adapter.tune(searchVector);

Prisma + pgvector Integration

import { PrismaClient } from '@prisma/client';
import sql from 'sql-template-tag';
import { withWarpVector } from 'warpvector/prisma';

const prisma = new PrismaClient().$extends(
  withWarpVector({ adapter, vectorField: "embedding", distanceOperator: "<=>" })
);

const results = await prisma.document.searchByVector({
  vector: rawVector, topK: 10, where: sql`category = 'science'`
});

๐Ÿงฉ Feature Overview

Category Features
Core Transforms IntentAdapter, LoraIntentAdapter, ProjectionAdapter
Neural Networks MlpAdapter (WASM), Non-linear activations (ReLU, Sigmoid, Tanh)
Online Learning WhiteningAdapter (PCA), SoftWhiteningAdapter (Inverse Diffusion)
Quantization Int8 scalar (4ร— compression), Binary (32ร— compression)
Reranking ColBERT/Late Interaction (WASM), TimeReversalReranker, MultipathScatteringReranker
Hybrid Search Reciprocal Rank Fusion (RRF), Relative Score Fusion (RSF)
Training InfoNCE, Triplet Loss, MigrationTrainer (Adam optimizer, edge-ready)
Auto-ML IntentMatrixFactory (auto-generate intent matrices from samples)
Advanced Task Arithmetic (model merging), VSA (Vector Symbolic Architecture), Federated Learning
Integrations Prisma + pgvector, LangChain, LlamaIndex
Runtime Zero dependencies, WASM/SIMD, Cloudflare Workers / Bun / Node.js

๐Ÿ” Debugging & Observability

// Inspect pipeline structure
console.log(pipeline.inspect());
// Pipeline [1536-dim]
//   Step 0: MlpAdapter
//   Step 1: IntentAdapter
//   Final: QuantizationAdapter

// Debug each step's intermediate output
const debug = pipeline.dryRun(testVector, { intent: "tech" });
debug.forEach(r => console.log(`${r.step}: dim=${r.output.length}, ${r.durationMs.toFixed(2)}ms`));

// Enable metrics collection
pipeline.metrics.enable();
pipeline.run(vector, { intent: "tech" });
console.log(pipeline.metrics.getMetrics());
// { totalRuns: 1, avgRunDurationMs: 0.12, avgStepDurationMs: { MlpAdapter: 0.05, ... } }

๐Ÿ“š Documentation

# Topic Description
0 Edge Quickstart Deploy on Cloudflare Workers / Vercel Edge
0.5 Auto-Learning Guide Build self-optimizing search pipelines
1 Core Adapters IntentAdapter, ProjectionAdapter, LoRA
2 Neural Networks MLP inference with WASM
3 Whitening / PCA Online anisotropy correction
4 Quantization Int8 (4ร—) and Binary (32ร—) compression
5 ColBERT WASM-accelerated late interaction
6 Hybrid Search RRF & RSF fusion
7 Trainers InfoNCE, Triplet, Online learning
8 Integrations LangChain, Prisma, LlamaIndex
9 Serialization State persistence & restoration
10 Projection & Migration Dimension reduction & model migration
11 Task Arithmetic Zero-overhead model merging
12 VSA Vector Symbolic Architecture
13 Feedback & Federated FeedbackCollector + FedAvg
14 Inverse Diffusion Semantic sharpening
15 Time-Reversal Reranker Wave-inspired reranking
16 Multipath Scattering Random-walk hub detection
17 IntentMatrixFactory Auto-generate intent matrices from samples
C1 E-commerce Search Cookbook Intent-based routing
C2 Pinecone RAG Cookbook Cost-efficient RAG
C3 Cloudflare Edge Cookbook Edge inference
โ€” API Reference Full API documentation
โ€” Troubleshooting Common issues & solutions
โ€” Migration Guide v0.1 โ†’ v0.2 upgrade guide

๐Ÿ“ Mathematical Background

Given a base embedding vector $\mathbf{x} \in \mathbb{R}^d$, WarpVector applies an affine map:

$$\mathbf{x}' = \sigma(\mathbf{W}_I \mathbf{x} + \mathbf{b}_I)$$

  • $\mathbf{W}_I \in \mathbb{R}^{d \times d}$: Intent transformation matrix (rotation, scaling, shearing)
  • $\mathbf{b}_I \in \mathbb{R}^d$: Intent bias vector (translation)
  • $\sigma$: Non-linear activation function (ReLU, Sigmoid, Tanh)

Computational complexity is $\mathcal{O}(d^2)$ (or $\mathcal{O}(d \cdot r)$ with LoRA), optimized via WASM and Float32Array memory alignment for sub-millisecond inference on edge devices.


โ˜๏ธ Cloudflare Vectorize Integration

import { WarpPipeline, VectorDBAdapter } from "warpvector";

// Upsert warped vectors
const records = documents.map((doc, i) =>
  VectorDBAdapter.toVectorizeRecord(`doc-${i}`, pipeline.run(doc.embedding), { title: doc.title })
);
await env.VECTORIZE_INDEX.upsert(records);

// Query with intent warping
const { vector, options } = VectorDBAdapter.toVectorizeQuery(
  pipeline.run(queryEmbedding),
  10,
  { returnMetadata: true }
);
const results = await env.VECTORIZE_INDEX.query(vector, options);

Also supports pgvector, Pinecone, and Redis out of the box.


๐Ÿ“Š OpenTelemetry-Compatible Tracing

import { WarpTracer, IntentAdapter } from "warpvector";

const tracer = new WarpTracer();
const adapter = new IntentAdapter(768);

// Trace any operation
const warped = tracer.trace("intent.tune", { intent: "tech", dim: 768 }, () => {
  return adapter.tune(vector, "tech");
});

// Get metrics
const metrics = tracer.getMetrics();
// { totalCalls: 1, avgLatencyMs: 0.12, operationCounts: { "intent.tune": 1 } }

Zero-dependency โ€” works without @opentelemetry/api installed.


๐Ÿค Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

๐Ÿ“„ License

MIT License