@damoxing/datasource-manager

中文文档: README.zh-CN.md

HTTP-agnostic multi-datasource lifecycle manager.

It owns:

• datasource config loading
• datasource status tracking
• generic route-key resolution, with jgbh compatibility
• heartbeat
• pool recovery
• one-shot retry for connection errors
• graceful shutdown

The core manager is not bound to HTTP. It can run inside Express, a worker, a CLI, or another Node.js service.

Database adapters are included for Oracle, DM, PostgreSQL, GaussDB/openGauss, and Kingbase. Drivers are optional peer dependencies and are loaded only when the corresponding adapter is used.

Adapter Contract

An adapter instance should expose:

{
    isOracle: Boolean,
    adapterType: String,
    initialize: async () => {},
    close: async () => {},
    all: async (sql, params) => [],
    get: async (sql, params) => row,
    run: async (sql, params) => result,
    exec: async (sql) => {},
    transaction: async (work) => result
}

withConnection(callback) is optional.

Usage

CommonJS:

const {
    DataSourceManager,
    createDefaultAdapterFactories
} = require('@damoxing/datasource-manager');

const manager = new DataSourceManager({
    config,
    logger,
    shutdownSignals: ['SIGTERM'],
    adapterFactories: createDefaultAdapterFactories(logger),
});

await manager.initialize();

const db = manager.getByJgbh('1001');
const row = await db.get('SELECT 1 AS health_check');

await manager.closeAll();

ESM:

import {
    DataSourceManager,
    createDefaultAdapterFactories
} from '@damoxing/datasource-manager';

const manager = new DataSourceManager({
    config,
    logger,
    adapterFactories: createDefaultAdapterFactories(logger),
});

Use getByRouteKey() for new non-business-specific code:

const db = manager.getByRouteKey('tenant-a');

getByJgbh() remains as a compatibility wrapper.

Health Output

getHealth() returns a stable, sanitized schema:

{
    generatedAt,
    initialized,
    strictRouting,
    defaultDatasourceId,
    routeCount,
    heartbeat: {
        enabled,
        running,
        intervalMs
    },
    shutdownHooks: {
        installed,
        signals
    },
    summary: {
        total,
        ready,
        failed,
        unhealthy,
        byStatus: {
            configured,
            initializing,
            ready,
            unhealthy,
            failed,
            recovering,
            closed
        }
    },
    datasources: [
        {
            id,
            type,
            status,
            jgbhCount,
            routeKeyCount,
            isDefault,
            lastHeartbeatAt,
            lastReadyAt,
            lastRecoverAt,
            lastStatusChangeAt,
            lastError
        }
    ]
}

Datasource config is never included in health output, so passwords and connect strings are not exposed.

Logging

Datasource errors are logged with structured context as the second logger argument:

{
    datasourceId,
    type,
    jgbh,
    jgbhList,
    routeKeys,
    status,
    lastError,
    error
}

SQL text logging is enabled by default. SQL parameter logging is disabled by default to avoid leaking production secrets.

Use adapter options to control parameter logging:

const adapterFactories = createDefaultAdapterFactories({
    logger,
    logSql: true,
    logParams: 'redacted',
    redactKeys: ['password', 'token', 'secret'],
    redactValue: '[REDACTED]'
});

logParams accepts:

• false or 'off': do not log SQL parameters
• true or 'redacted': log parameters with sensitive keys redacted
• 'raw': log raw parameters for local debugging only

Graceful Shutdown

Use shutdownSignals to close all pools when the process receives a signal:

const manager = new DataSourceManager({
    shutdownSignals: ['SIGTERM', 'SIGINT'],
    exitOnShutdownSignal: true,
    adapterFactories,
    config
});

closeAll() stops heartbeat timers, removes shutdown hooks, closes adapters, and marks datasources as closed.

Config Shape

{
    "strict_routing": true,
    "default_datasource": "oracle_main",
    "datasources": [
        {
            "id": "oracle_main",
            "type": "oracle",
            "jgbh_list": ["1001"],
            "route_keys": ["tenant-a"],
            "config": {}
        }
    ]
}

Pool Governance

Use normalized pool options at datasource level or under config.pool:

{
    "id": "pg_main",
    "type": "pg",
    "pool": {
        "minPoolSize": 1,
        "maxPoolSize": 10,
        "acquireTimeoutMs": 3000,
        "connectTimeoutMs": 3000,
        "idleTimeoutMs": 60000,
        "maxLifetimeMs": 1800000,
        "keepaliveMs": 30000
    },
    "config": {}
}

The manager maps supported options to each driver and logs unsupported options with datasource context. Invalid values, such as minPoolSize > maxPoolSize, fail during config build.

applyPoolGovernance(type, config, pool) is exported for tests and diagnostics.

Metrics

getMetrics() returns counters, gauges, and latency summaries without SQL text, parameters, passwords, or connection strings:

const metrics = manager.getMetrics();

Metrics currently track initialization, heartbeat, recovery, query success/failure, connection errors, retries, and transaction connection errors.

Runtime Governance

Datasources can be changed at runtime:

await manager.addDatasource({ id: 'tenant_a', type: 'pg', route_keys: ['TENANT_A'], config: {} });
await manager.updateDatasource('tenant_a', { id: 'tenant_a', type: 'pg', route_keys: ['TENANT_A'], config: {} });
await manager.removeDatasource('tenant_a');
await manager.reloadConfig(nextConfig);

updateDatasource() initializes and pings the replacement before closing the old pool. If replacement initialization fails, the old datasource remains active.

Read/Write Groups

Group routing is HTTP-agnostic:

{
    "groups": {
        "tenant_a": {
            "write": "tenant_a_master",
            "read": [
                "tenant_a_read_1",
                { "id": "tenant_a_read_2", "weight": 2 }
            ],
            "strategy": "round-robin",
            "fallbackToWrite": true
        }
    }
}

const readDb = manager.getReadHandle('tenant_a');
const writeDb = manager.getWriteHandle('tenant_a');

Read strategies are round-robin, random, weighted, and first-ready. Unhealthy read replicas are skipped.

Config Secrets

Config values support environment interpolation, environment references, secret resolver hooks, and encrypted values:

const manager = new DataSourceManager({
    env: process.env,
    secretResolver: async key => loadSecret(key),
    decryptor: async value => decrypt(value),
    config
});

{
    "password": "env:DB_PASSWORD",
    "token": "secret:database/token",
    "connectString": "postgres://app:${DB_PASSWORD}@localhost/db",
    "encrypted": "ENC(ciphertext)"
}

Resolved secrets are redacted from health output, metrics output, structured datasource error state, and manager logs.

Retry Rule

Connection-like errors may be retried once after pool recovery.

Transactions are not replayed automatically. If a transaction fails with a connection error, the manager rebuilds the pool and rethrows the original error.

npm Packaging

This package is authored in TypeScript under src/ and builds both module formats:

• CommonJS: dist/cjs/index.js
• ESM: dist/esm/index.mjs
• Types: dist/types/index.d.ts

Build locally before publishing:

npm install
npm run release:check

The ESM output is compiled from TypeScript with esbuild. Root and adapter aggregate imports keep database drivers lazy, so importing the package does not immediately load oracledb, dmdb, or pg.

Built-in database drivers are declared as optional peer dependencies:

• oracledb for Oracle
• dmdb for DM
• pg for PostgreSQL, GaussDB/openGauss, and Kingbase

Importing the package root does not immediately load these drivers. Accessing adapter classes or creating an adapter through createDefaultAdapterFactories() loads only the driver needed by that datasource type.

See RELEASE.md for semver, registry, token, provenance, and final publish checklist guidance.

See ROADMAP.md for the enterprise datasource framework backlog and priority order.