@loro-dev/flock

TypeScript bindings for the Flock Conflict-free Replicated Data Type (CRDT). Flock stores JSON-like values at composite keys, keeps causal metadata, and synchronises peers through mergeable export bundles. This package wraps the core MoonBit implementation and exposes a typed Flock class for JavaScript runtimes (Node.js ≥18, modern browsers, and workers).

Installation

pnpm add @loro-dev/flock
# or
npm install @loro-dev/flock
# or
yarn add @loro-dev/flock

The library ships ESM, CommonJS, and TypeScript declaration files. It has no runtime dependencies beyond the standard Web Crypto API for secure peer identifiers (falls back to Math.random when unavailable).

Quick Start

import { Flock } from "@loro-dev/flock";

// Each replica uses a stable 32-byte peer id to maintain version vectors.
const peerId = crypto.getRandomValues(new Uint8Array(32));
const store = new Flock(peerId);

store.put(["users", "42"], { name: "Ada", online: true });
console.log(store.get(["users", "42"]));

// Share incremental updates with a remote replica.
const bundle = store.exportJson();

// On another node, merge and materialise the same data.
const other = new Flock();
other.importJson(bundle);
other.merge(store);

// Subscribe to local or remote mutations.
const unsubscribe = store.subscribe(({ source, events }) => {
  for (const { key, value } of events) {
    console.log(`[${source}]`, key, value ?? "<deleted>");
  }
});

store.delete(["users", "42"]);
unsubscribe();

Replication Workflow

• Call exportJson() to serialise local changes since an optional version vector.
• Distribute bundles via your transport of choice (HTTP, WebSocket, etc.).
• Apply remote bundles with importJson() and reconcile replicas using merge().
• Track causal progress with version() and getMaxPhysicalTime() when orchestrating incremental syncs.
• Use Flock.checkConsistency() in tests to assert that two replicas converged to the same state.

Type Basics

• Value: JSON-compatible data (string, number, boolean, null, nested arrays/objects).
• KeyPart: string | number | boolean. Keys are arrays of parts (e.g. ["users", 42]). Invalid keys raise at runtime.
• ExportRecord: { c: string; d?: Value } – CRDT payload with hybrid logical clock data.
• ExportBundle: Record<string, ExportRecord> mapping composite keys to last-writer metadata.
• VersionVector: Record<string, { physicalTime: number; logicalCounter: number }> indexed by hex peer identifiers.
• ScanRow: { key: KeyPart[]; raw: ExportRecord; value?: Value } returned by scan().
• EventBatch: { source: string; events: Array<{ key: KeyPart[]; value?: Value }> } emitted to subscribers.

All types are exported from the package entry point for use in TypeScript projects.

API Reference

Constructor

new Flock(peerId?: Uint8Array) – creates a replica. When omitted, a random 256-bit peer id is generated. The id persists only in memory; persist it yourself for durable replicas.

Static Members

• Flock.fromJson(bundle: ExportBundle, peerId: Uint8Array): Flock – instantiate directly from a full snapshot bundle.
• Flock.checkConsistency(a: Flock, b: Flock): boolean – deep equality check useful for tests; returns true when both replicas expose the same key/value pairs and metadata.

Replica Management

• setPeerId(peerId: Uint8Array): void – replace the current peer id. Use cautiously; changing ids affects causality tracking.
• peerId(): Uint8Array – returns the 32-byte identifier for the replica.
• getMaxPhysicalTime(): number – highest physical timestamp observed by this replica (same units as the timestamps you pass to now, e.g. Date.now() output). Helpful for synchronising clocks and diagnosing divergence.
• version(): VersionVector – current version vector including logical counters per peer.
• checkInvariants(): void – throws if internal CRDT invariants are violated. Intended for assertions in tests or diagnostics, not for production control flow.

Mutations

• put(key: KeyPart[], value: Value, now?: number): void – write a JSON value. The optional now overrides the physical time (numeric timestamp such as Date.now() output) used for the replica’s hybrid logical clock. Invalid keys or non-finite timestamps throw.
• set(key: KeyPart[], value: Value, now?: number): void – alias of put for frameworks that prefer “set” terminology.
• delete(key: KeyPart[], now?: number): void – tombstone a key. The optional time override follows the same rules as put.
• putMvr(key: KeyPart[], value: Value, now?: number): void – attach a value to the key’s Multi-Value Register. Unlike put, concurrent writes remain alongside each other.

Reads

• get(key: KeyPart[]): Value | undefined – fetch the latest visible value. Deleted keys resolve to undefined.
• getMvr(key: KeyPart[]): Value[] – read all concurrent values associated with a key’s Multi-Value Register (empty array when unset).
• kvToJson(): ExportBundle – snapshot of every key/value pair including tombstones. Useful for debugging or serialising the entire store.
• scan(options?: ScanOptions): ScanRow[] – in-order range scan. Supports:
  • start / end: { kind: "inclusive" | "exclusive"; key: KeyPart[] } or { kind: "unbounded" }.
  • prefix: restrict results to keys beginning with the provided prefix. Results include the materialised value (if any) and raw CRDT record.

Replication

• exportJson(from?: VersionVector): ExportBundle – export causal updates. Provide a VersionVector from a remote replica to stream only novel changes; omit to export the entire dataset.
• importJson(bundle: ExportBundle): void – apply a bundle received from another replica. Invalid payloads throw.
• merge(other: Flock): void – merge another replica instance directly (both replicas end up with the joined state).

Events

• subscribe(listener: (batch: EventBatch) => void): () => void – register for mutation batches. Each callback receives the source ("local" for writes on this replica, peer id for remote batches when available) and an ordered list of events. Return value unsubscribes the listener.

Utilities

• exportJson, importJson, kvToJson, and fromJson all work with plain JavaScript objects, so they serialise cleanly through JSON or structured clone.
• Keys are encoded using MoonBit’s memcomparable format; use simple scalars and natural ordering for predictable scans.

Error Handling

• Methods that accept keys validate them at runtime. Passing unsupported key parts (like objects or undefined) throws a TypeError.
• Passing malformed bundles or version vectors throws a TypeError or propagates an underlying runtime error from the native module.
• Always wrap replication IO in try/catch when dealing with untrusted data.

Testing Tips

• Use Flock.checkConsistency() to assert two replicas match after a sequence of operations.
• checkInvariants() is safe inside unit tests to catch bugs in integration layers.
• Vitest users can combine exportJson snapshots with expect to capture deterministic state transitions.

License

MIT © Loro contributors