JSPM

  • ESM via JSPM
  • ES Module Entrypoint
  • Export Map
  • Keywords
  • License
  • Repository URL
  • TypeScript Types
  • README
  • Created
  • Published
  • Downloads 22
  • Score
    100M100P100Q37517F
  • License MIT

A TypeScript-based database abstraction layer for IPFS/IPNS decentralized storage

Package Exports

  • weavestore

Readme

Weavestore

A TypeScript-based database abstraction layer that provides traditional database operations (CRUD) on top of the IPFS/IPNS decentralized storage network.

Features

  • 🗄️ Database-like operations on IPFS/IPNS
  • 🏗️ Hierarchical structure: Database → Schema → Table organization
  • 🔒 Type-safe TypeScript API
  • 📦 Support for Lighthouse (IPFS and IPNS)
  • 🆔 Configurable ID generation (Snowflake or UUID)
  • 💾 Built-in caching with LRU eviction
  • 🔄 Automatic retry and failover
  • 🚀 Built with Bun for optimal performance
  • 🔐 Better Auth adapter for decentralized authentication
  • 🔙 Backward compatible with legacy flat table structure

Installation

bun add weavestore

Or with npm:

npm install weavestore

Quick Start

Basic Usage (Legacy API - Still Supported)

import { WeaveStore } from 'weavestore';

// Initialize the SDK
const sdk = new WeaveStore({
    lighthouse: {
        apiKey: 'your-lighthouse-api-key',
    },
    ipns: {
        apiKey: 'your-lighthouse-ipns-api-key',
    },
});

await sdk.initialize();

// Create a table (uses default database and schema)
const usersTable = await sdk.createTable('users');

// Insert a record
const userId = await usersTable.insert({
  name: 'John Doe',
  email: 'john@example.com',
  age: 30,
});

// Query records
const users = await usersTable.find({ age: { $gte: 25 } });

// Update a record by ID
await usersTable.updateById(userId, { age: 31 });

// Update multiple records with filter
const updatedCount = await usersTable.update(
  { age: { $lt: 18 } },
  { status: 'minor' }
);

// Soft delete a record (default)
await usersTable.deleteById(userId);

// Hard delete a record (removes from Lighthouse storage)
await usersTable.deleteById(userId, true);

// Delete multiple records with filter
const deletedCount = await usersTable.delete({ status: 'inactive' });

Weavestore now supports a hierarchical structure: Database → Schema → Table

import { WeaveStore } from 'weavestore';

const sdk = new WeaveStore({
    lighthouse: { apiKey: 'your-lighthouse-api-key' },
    ipns: { apiKey: 'your-lighthouse-ipns-api-key' },
});

await sdk.initialize();

// Create a database
const myAppDb = await sdk.createDatabase('my_app');

// Create schemas within the database
const publicSchema = await myAppDb.createSchema('public');
const authSchema = await myAppDb.createSchema('auth');

// Create tables within schemas
const usersTable = await publicSchema.createTable('users');
const sessionsTable = await authSchema.createTable('sessions');

// Insert data
const userId = await usersTable.insert({
  name: 'John Doe',
  email: 'john@example.com',
});

// Access tables using path notation
const table = await sdk.getTableByPath('my_app.public.users');

// List all databases
const databases = await sdk.listDatabases();

// List schemas in a database
const schemas = await myAppDb.listSchemas();

// List tables in a schema
const tables = await publicSchema.listTables();

Hierarchical Structure

Weavestore organizes data in a three-tier hierarchy:

Database → Schema → Table → Records

Why Use Hierarchical Structure?

  • Multi-tenancy: Separate data for different tenants/applications
  • Logical Grouping: Organize related tables into schemas (e.g., 'public', 'auth', 'analytics')
  • Namespace Management: Avoid table name conflicts across different contexts
  • Better Organization: Scale to hundreds of tables without chaos

Database Operations

// Create a database
const database = await sdk.createDatabase('my_app', {
  cache: { enabled: true, ttl: 600000 },
  isolationLevel: 'read-committed',
});

// Get an existing database
const db = await sdk.getDatabase('my_app');

// List all databases
const databases = await sdk.listDatabases();

// Delete a database
await sdk.deleteDatabase('old_app'); // Soft delete
await sdk.deleteDatabase('old_app', true); // Hard delete

Schema Operations

// Create a schema
const schema = await database.createSchema('public', {
  validation: { enabled: true, strict: true },
  defaultTableSettings: { idStrategy: 'uuid' },
  namingConvention: { case: 'snake_case', prefix: 'tbl_' },
});

// Get an existing schema
const publicSchema = await database.getSchema('public');

// List all schemas
const schemas = await database.listSchemas();

// Batch load schemas
const [publicSchema, authSchema] = await database.batchGetSchemas(['public', 'auth']);

// Prefetch schemas for faster access
await database.prefetchSchemas(['public', 'auth']);

// Delete a schema
await database.deleteSchema('old_schema'); // Soft delete
await database.deleteSchema('old_schema', true); // Hard delete

Table Operations in Schema Context

// Create a table in a schema
const usersTable = await schema.createTable('users', {
  name: { type: 'string', required: true },
  email: { type: 'string', required: true },
  age: { type: 'number' },
});

// Get a table from a schema
const table = await schema.getTable('users');

// List all tables in a schema
const tables = await schema.listTables();

// Batch load tables
const [usersTable, postsTable] = await schema.batchGetTables(['users', 'posts']);

// Prefetch tables
await schema.prefetchTables(['users', 'posts']);

// Delete a table
await schema.deleteTable('old_table'); // Soft delete
await schema.deleteTable('old_table', true); // Hard delete

Path-Based Access

Access entities using intuitive path notation:

// Get table by full path
const usersTable = await sdk.getTableByPath('my_app.public.users');

// Also supports slash notation
const table = await sdk.getTableByPath('my_app/public/users');

// Get schema by path
const schema = await sdk.getSchemaByPath('my_app.public');

// Get database by path
const database = await sdk.getDatabaseByPath('my_app');

Backward Compatibility

The legacy flat table API still works and uses a default database and schema:

// These methods still work (with deprecation warnings)
const table = await sdk.createTable('users'); // Creates in __default__.public.users
const table = await sdk.getTable('users'); // Gets from __default__.public.users
const tables = await sdk.listTables(); // Lists tables from __default__.public

Configuration

SDK Configuration Options

const sdk = new WeaveStore({
    lighthouse: {
        apiKey: 'your-lighthouse-api-key',
        gateway: 'https://gateway.lighthouse.storage', // Optional custom gateway
    },
    ipns: {
        apiKey: 'your-lighthouse-ipns-api-key',
    },
    idStrategy: 'snowflake', // or 'uuid' (default: 'snowflake')
    cache: {
        enabled: true,
        ttl: 300000, // 5 minutes in milliseconds
        maxSize: 104857600, // 100MB in bytes
    },
    retry: {
        maxAttempts: 3,
        backoffMs: 1000,
    },
});

API Reference

SDK Methods

initialize()

Initialize the SDK and verify API connections.

await sdk.initialize();

Database Management

createDatabase(name: string, config?: DatabaseConfig)

Create a new database.

const database = await sdk.createDatabase('my_app', {
  cache: { enabled: true, ttl: 600000 },
  isolationLevel: 'read-committed',
});
getDatabase(name: string)

Get an existing database.

const database = await sdk.getDatabase('my_app');
listDatabases(options?: { limit?: number; offset?: number })

List all databases with optional pagination.

const databases = await sdk.listDatabases();
const page1 = await sdk.listDatabases({ limit: 10, offset: 0 });
deleteDatabase(name: string, hardDelete?: boolean)

Delete a database.

await sdk.deleteDatabase('old_app'); // Soft delete
await sdk.deleteDatabase('old_app', true); // Hard delete

Path-Based Access

getTableByPath(path: string)

Get a table using full path notation.

const table = await sdk.getTableByPath('my_app.public.users');
// Also supports slash notation
const table = await sdk.getTableByPath('my_app/public/users');
getSchemaByPath(path: string)

Get a schema using path notation.

const schema = await sdk.getSchemaByPath('my_app.public');
getDatabaseByPath(path: string)

Get a database using path notation.

const database = await sdk.getDatabaseByPath('my_app');

Legacy Methods (Deprecated but Supported)

createTable<T>(name: string, schema?: Schema<T>) ⚠️ Deprecated

Create a new table in the default database and schema.

interface User {
  name: string;
  email: string;
  age: number;
}

const usersTable = await sdk.createTable<User>('users');
// Recommended: Use database.schema.createTable() instead
getTable<T>(name: string) ⚠️ Deprecated

Get an existing table from the default database and schema.

const usersTable = await sdk.getTable<User>('users');
// Recommended: Use database.schema.getTable() or sdk.getTableByPath() instead
listTables() ⚠️ Deprecated

List all tables in the default database and schema.

const tables = await sdk.listTables();
// Recommended: Use database.schema.listTables() instead
deleteTable(name: string) ⚠️ Deprecated

Delete a table from the default database and schema.

await sdk.deleteTable('users');
// Recommended: Use database.schema.deleteTable() instead

Database Class Methods

createSchema(name: string, config?: SchemaConfig)

Create a new schema within the database.

const schema = await database.createSchema('public', {
  validation: { enabled: true, strict: true },
  defaultTableSettings: { idStrategy: 'uuid' },
});

getSchema(name: string)

Get an existing schema.

const schema = await database.getSchema('public');

listSchemas(options?: { limit?: number; offset?: number })

List all schemas with optional pagination.

const schemas = await database.listSchemas();

batchGetSchemas(names: string[])

Load multiple schemas in parallel.

const [publicSchema, authSchema] = await database.batchGetSchemas(['public', 'auth']);

prefetchSchemas(names: string[])

Prefetch schema metadata for faster subsequent access.

await database.prefetchSchemas(['public', 'auth']);

deleteSchema(name: string, hardDelete?: boolean)

Delete a schema.

await database.deleteSchema('old_schema'); // Soft delete
await database.deleteSchema('old_schema', true); // Hard delete

getMetadata()

Get database metadata.

const metadata = database.getMetadata();
console.log(metadata.name, metadata.schemas);

refresh(cascade?: boolean)

Refresh database metadata from IPNS.

await database.refresh(); // Refresh just database
await database.refresh(true); // Refresh database and all children

Schema Class Methods

createTable<T>(name: string, schema?: TableSchema<T>)

Create a new table within the schema.

const usersTable = await schema.createTable('users', {
  name: { type: 'string', required: true },
  email: { type: 'string', required: true },
});

getTable<T>(name: string)

Get an existing table.

const table = await schema.getTable('users');

listTables(options?: { limit?: number; offset?: number })

List all tables with optional pagination.

const tables = await schema.listTables();

batchGetTables<T>(names: string[])

Load multiple tables in parallel.

const [usersTable, postsTable] = await schema.batchGetTables(['users', 'posts']);

prefetchTables(names: string[])

Prefetch table metadata for faster subsequent access.

await schema.prefetchTables(['users', 'posts']);

deleteTable(name: string, hardDelete?: boolean)

Delete a table.

await schema.deleteTable('old_table'); // Soft delete
await schema.deleteTable('old_table', true); // Hard delete

getMetadata()

Get schema metadata.

const metadata = schema.getMetadata();
console.log(metadata.name, metadata.tables);

refresh(cascade?: boolean)

Refresh schema metadata from IPNS.

await schema.refresh(); // Refresh just schema
await schema.refresh(true); // Refresh schema and all tables

Table Methods

insert(data: T)

Insert a single record.

const userId = await usersTable.insert({
  name: 'John Doe',
  email: 'john@example.com',
  age: 30,
});

insertMany(data: T[])

Insert multiple records in a single operation.

const userIds = await usersTable.insertMany([
  { name: 'Alice', email: 'alice@example.com', age: 25 },
  { name: 'Bob', email: 'bob@example.com', age: 30 },
]);

findById(id: string)

Find a record by ID.

const user = await usersTable.findById(userId);

findAll(options?: QueryOptions)

Find all records.

const users = await usersTable.findAll({
  limit: 10,
  offset: 0,
  includeDeleted: false,
  sortBy: 'age',
  sortOrder: 'desc',
});

find(filter: Filter<T>, options?: QueryOptions)

Find records matching a filter.

const adults = await usersTable.find({ age: { $gte: 18 } });

findOne(filter: Filter<T>)

Find the first record matching a filter.

const user = await usersTable.findOne({ email: 'john@example.com' });

count(filter?: Filter<T>)

Count records matching a filter.

const adultCount = await usersTable.count({ age: { $gte: 18 } });

updateById(id: string, data: Partial<T>)

Update a record by ID.

await usersTable.updateById(userId, { age: 31 });

update(filter: Filter<T>, data: Partial<T>)

Update multiple records matching a filter. Returns the number of updated records.

// Update all inactive users to active
const updatedCount = await usersTable.update(
  { status: 'inactive' },
  { status: 'active' }
);
console.log(`Updated ${updatedCount} users`);

deleteById(id: string, hardDelete?: boolean)

Delete a record by ID.

  • Soft delete (default): Marks the record as deleted but keeps it in storage
  • Hard delete: Actually removes the file from Lighthouse storage (only works for Annual Plan files)
// Soft delete (default)
await usersTable.deleteById(userId);

// Hard delete (removes from storage)
await usersTable.deleteById(userId, true);

delete(filter: Filter<T>, hardDelete?: boolean)

Delete multiple records matching a filter. Returns the number of deleted records.

// Soft delete all inactive users
const deletedCount = await usersTable.delete({ status: 'inactive' });
console.log(`Deleted ${deletedCount} users`);

// Hard delete all users over 65
const hardDeletedCount = await usersTable.delete({ age: { $gt: 65 } }, true);

healthCheck()

Check the health of IPFS and IPNS connections.

const health = await sdk.healthCheck();
console.log(health); // { ipfs: true, ipns: true, timestamp: 1234567890 }

clearCache()

Clear the cache.

sdk.clearCache();

Table Methods

insert(data: T)

Insert a single record.

const userId = await usersTable.insert({
  name: 'John Doe',
  email: 'john@example.com',
  age: 30,
});

insertMany(data: T[])

Insert multiple records.

const userIds = await usersTable.insertMany([
  { name: 'John', email: 'john@example.com', age: 30 },
  { name: 'Jane', email: 'jane@example.com', age: 25 },
]);

findById(id: string)

Find a record by ID.

const user = await usersTable.findById(userId);

findAll(options?: QueryOptions)

Find all records with optional pagination.

const users = await usersTable.findAll({
  limit: 10,
  offset: 0,
  includeDeleted: false,
});

find(filter: Filter<T>, options?: QueryOptions)

Find records matching a filter.

const adults = await usersTable.find(
  { age: { $gte: 18 } },
  { limit: 10, sortBy: 'age', sortOrder: 'desc' }
);

findOne(filter: Filter<T>)

Find the first record matching a filter.

const user = await usersTable.findOne({ email: 'john@example.com' });

count(filter?: Filter<T>)

Count records matching a filter.

const adultCount = await usersTable.count({ age: { $gte: 18 } });

updateById(id: string, data: Partial<T>)

Update a record by ID.

await usersTable.updateById(userId, { age: 31 });

deleteById(id: string)

Delete a record by ID (soft delete).

await usersTable.deleteById(userId);

getMetadata()

Get table metadata.

const metadata = usersTable.getMetadata();

refresh()

Refresh table metadata from IPNS.

await usersTable.refresh();

Query Filtering

The SDK supports MongoDB-style query operators for powerful and flexible querying:

Comparison Operators

  • $eq: Equal to
  • $ne: Not equal to
  • $gt: Greater than
  • $gte: Greater than or equal to
  • $lt: Less than
  • $lte: Less than or equal to

Array Operators

  • $in: Value is in array
  • $nin: Value is not in array

Pattern Matching (NEW)

  • $regex: Regular expression pattern matching (like SQL LIKE)
  • $options: Regex options ('i' for case-insensitive, 'g' for global, 'm' for multiline)

Field Operators (NEW)

  • $exists: Check if field exists

Logical Operators (NEW)

  • $or: Logical OR - matches if any condition is true
  • $and: Logical AND - matches if all conditions are true
  • $not: Logical NOT - inverts the condition

Basic Examples

// Find users older than 25
const users = await usersTable.find({ age: { $gt: 25 } });

// Find users between 18 and 65
const users = await usersTable.find({
  age: { $gte: 18, $lte: 65 },
});

// Find users with specific emails
const users = await usersTable.find({
  email: { $in: ['john@example.com', 'jane@example.com'] },
});

// Find users not named John
const users = await usersTable.find({
  name: { $ne: 'John' },
});

Advanced Search Examples (NEW)

// Pattern matching - find users whose name contains "john" (case-insensitive)
const users = await usersTable.find({
  name: { $regex: 'john', $options: 'i' }
});

// Search across multiple fields with OR
const searchQuery = 'john';
const results = await usersTable.find({
  $or: [
    { name: { $regex: searchQuery, $options: 'i' } },
    { username: { $regex: searchQuery, $options: 'i' } },
    { email: { $regex: searchQuery, $options: 'i' } }
  ]
});

// Complex nested logic with AND and OR
const users = await usersTable.find({
  $and: [
    { age: { $gte: 25 } },
    { city: 'NYC' },
    {
      $or: [
        { status: 'active' },
        { status: 'pending' }
      ]
    }
  ]
});

// NOT operator - find users NOT in NYC
const users = await usersTable.find({
  $not: { city: 'NYC' }
});

// Check field existence
const usersWithEmail = await usersTable.find({
  email: { $exists: true }
});

Advanced Pagination (NEW)

// Using offset
const page1 = await usersTable.find({}, { limit: 10, offset: 0 });
const page2 = await usersTable.find({}, { limit: 10, offset: 10 });

// Using skip (alias for offset)
const page1 = await usersTable.find({}, { limit: 10, skip: 0 });
const page2 = await usersTable.find({}, { limit: 10, skip: 10 });

// Dynamic pagination
const page = 2;
const limit = 10;
const users = await usersTable.find(
  {},
  { skip: (page - 1) * limit, limit: limit }
);

// Pagination with sorting and filtering
const users = await usersTable.find(
  { status: 'active' },
  {
    sortBy: 'createdAt',
    sortOrder: 'desc',
    skip: 20,
    limit: 10
  }
);
// Search for users with complex criteria
const searchQuery = 'john';
const page = 1;
const limit = 20;

const users = await usersTable.find(
  {
    $and: [
      // Must have email
      { email: { $exists: true } },
      
      // Must be 18 or older
      { age: { $gte: 18 } },
      
      // Must not be banned
      { $not: { status: 'banned' } },
      
      // Search in name or username
      {
        $or: [
          { name: { $regex: searchQuery, $options: 'i' } },
          { username: { $regex: searchQuery, $options: 'i' } }
        ]
      },
      
      // Must be in NYC or SF
      {
        $or: [
          { city: 'NYC' },
          { city: 'SF' }
        ]
      }
    ]
  },
  {
    skip: (page - 1) * limit,
    limit: limit,
    sortBy: 'name',
    sortOrder: 'asc'
  }
);

For more advanced search and pagination examples, see the Advanced Search and Pagination Guide.

Better Auth Integration

Weavestore includes a built-in adapter for Better Auth, enabling decentralized authentication storage on IPFS/IPNS.

Quick Setup

import { IPFSStorageSDK, createBetterAuthAdapter } from 'weavestore';
import { betterAuth } from 'better-auth';

// Initialize SDK
const sdk = new IPFSStorageSDK({
  lighthouse: {
    apiKey: process.env.LIGHTHOUSE_API_KEY!,
  },
  ipns: {
    apiKey: process.env.LIGHTHOUSE_IPNS_API_KEY!,
  },
  idStrategy: 'uuid',
});

await sdk.initialize();

// Create Better Auth adapter
const adapter = createBetterAuthAdapter({
  sdk,
  tablePrefix: 'auth_',  // Optional: prefix for all tables
  usePlural: true,       // Optional: use plural table names
});

// Use with Better Auth
export const auth = betterAuth({
  database: adapter,
  emailAndPassword: {
    enabled: true,
  },
  socialProviders: {
    google: {
      clientId: process.env.GOOGLE_CLIENT_ID!,
      clientSecret: process.env.GOOGLE_CLIENT_SECRET!,
    },
  },
});

Features

  • ✅ Full CRUD operations for all Better Auth models
  • ✅ Automatic table creation
  • ✅ Complex queries with filtering
  • ✅ Pagination and sorting
  • ✅ Field selection for optimized queries
  • ✅ Decentralized storage on IPFS/IPNS
  • ✅ No migrations needed

Supported Operations

The adapter automatically handles:

  • User management (create, read, update, delete)
  • Session management
  • OAuth account linking
  • Verification tokens
  • All Better Auth models

Example

See the Better Auth integration example for a complete working example.

Documentation

For detailed documentation, see Better Auth Adapter README.

Error Handling

The SDK throws IPFSStorageError with specific error codes:

import { IPFSStorageError, ErrorCode } from 'weavestore';

try {
  await sdk.initialize();
} catch (error) {
  if (error instanceof IPFSStorageError) {
    switch (error.code) {
      case ErrorCode.AUTHENTICATION_FAILED:
        console.error('Authentication failed:', error.message);
        break;
      case ErrorCode.TABLE_NOT_FOUND:
        console.error('Table not found:', error.message);
        break;
      case ErrorCode.NETWORK_ERROR:
        console.error('Network error:', error.message);
        break;
      // ... handle other error codes
    }
  }
}

Error Codes

  • AUTHENTICATION_FAILED: API authentication failed
  • TABLE_NOT_FOUND: Table does not exist
  • TABLE_ALREADY_EXISTS: Table already exists
  • RECORD_NOT_FOUND: Record does not exist
  • NETWORK_ERROR: Network request failed
  • TIMEOUT: Operation timed out
  • VALIDATION_ERROR: Input validation failed
  • IPFS_UPLOAD_FAILED: IPFS upload failed
  • IPNS_PUBLISH_FAILED: IPNS publish failed
  • RATE_LIMIT_EXCEEDED: API rate limit exceeded

Performance Considerations

Caching

The SDK includes built-in caching with LRU eviction:

  • Record data is cached after retrieval
  • Record indexes are cached aggressively
  • Cache is automatically invalidated on updates
  • Configure cache size and TTL based on your needs

Batch Operations

Use insertMany() for bulk inserts to reduce IPNS updates:

// Good: Single IPNS update
await usersTable.insertMany([user1, user2, user3]);

// Less efficient: Multiple IPNS updates
await usersTable.insert(user1);
await usersTable.insert(user2);
await usersTable.insert(user3);

Scalability Limits

  • Records per table: Recommended max 10,000 records
  • Record size: Recommended max 1MB per record
  • Concurrent operations: Reads are parallel, writes are serialized per table

Getting API Keys

To use this SDK, you need Lighthouse API keys:

  1. Visit https://files.lighthouse.storage/
  2. Login/Sign up for a free account
  3. Generate an API key from the API Key section
  4. Update your .env file with the key

📚 Detailed Guide: HOW_TO_GET_API_KEYS.md

Troubleshooting

Authentication Errors

Ensure your API keys are correct:

# Lighthouse (for IPFS uploads)
export LIGHTHOUSE_API_KEY="your-api-key"

# Lighthouse IPNS (for IPNS operations)
export LIGHTHOUSE_IPNS_API_KEY="your-ipns-api-key"

Network Errors

The SDK automatically retries failed requests with exponential backoff. If you're experiencing persistent network errors:

  1. Check your internet connection
  2. Verify API service status
  3. Increase retry attempts in configuration

Gateway Failures

The SDK automatically fails over to alternative IPFS gateways if the primary gateway is unavailable.

Development

Building

bun run build

Type Checking

bun run typecheck

Testing

bun test

License

MIT

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

Support

For issues and questions, please open an issue on GitHub.