CustomMetrics

Low cost, fast, simple, scalable metrics for AWS

CustomMetrics is a NodeJS library to emit and query custom metrics for AWS apps.

CustomMetrics is under active development and is not (yet) ready for production use. This README documentation is incomplete. All feedback is welcome.

Background

AWS CloudWatch offers metrics to monitor specific aspects of your apps that are not covered by the default AWS infrastructure metrics.

Unfortunately, the AWS "custom" metrics can be very expensive. If updated or queried regularly. Each each custom metric will cost up to $5 per metric per year with additional costs for querying. If you have many metrics or high dimensionality on your metrics, this can lead to a very large CloudWatch Metrics bill.

CustomMetrics provides cost effective metrics that are up to 1000 times cheaper and 10 times faster than standard CloudWatch metrics.

CustomMetrics achieves these savings by supporting only "latest" period metrics. i.e. last day, last month, last hour etc. This enables each metric to be saved, stored and queried with minimal cost.

CustomMetrics Features

• Simple one line API to emit metrics from any NodeJS TypeScript or JavaScript app.
• Similar metric model to AWS supporting namespaces, metrics, dimensions, statistics and intervals.
• Computes statistics for: average, min, max, count and sum.
• Computes P value statistics with configurable P value resolution.
• Supports a default metric intervals of: last 5 mins, hour, day, week, month and year.
• Configurable custom intervals for higher or different metric intervals.
• Fast and flexible query API to fetch by namespace, metric and dimensions.
• Query API can return data points or aggregate metric as a single statistic.
• Scalable to support many simultaneous clients emitting metrics.
• Stores data in any existing DynamoDB table and can co-exist with existing app data.
• Supports multiple services, apps, namespaces and metrics in a single DynamoDB table.
• Extremely fast initialization time.
• Written in TypeScript with full TypeScript support.
• Clean, readable small code base (<1K lines).
• SenseDeep support for visualizing and graphing metrics.

Database

CustomMetrics stores each metric in a single, compressed DynamoDB item.

Quick Tour

Install the library using npm or yarn.

npm i CustomMetrics

Import the CustomMetrics library. If you are not using ES modules or TypeScript, use require to import the library.

import {CustomMetrics} from 'CustomMetrics'

Next create and configure the CustomMetrics instance.

// Create OneTable instance for your DynamoDB table
const metrics = new CustomMetrics({
    client: dynamoDbClient,
    owner: 'my-service',
    tableName: 'MyTable',
    primaryKey: 'pk',
    sortKey: 'sk',
})

Metrics emitted by an instance will be scoped and "owned" by the owner property you specify. This is typically a service, application or account name. CustomMetric instances with different owners are isolated from each other and their metrics will not interfere with each other. If omitted, the owner defaults to 'account'.

Metrics are stored in the DynamoDB database referenced by the dynamodDbClient instance which is an AWS V3 DynamoDB Document Client instance.

import {DynamoDBClient} from '@aws-sdk/client-dynamodb'

const dynamoDbClient = new DynamoDBClient()

OneTable

Alternatively, if you are using OneTable you can construct CustomMetrics using your OneTable instance. In this case, the table name and primary/sort keys are inferred from the OneTable instance.

// Create OneTable instance for your DynamoDB table
const metrics = new CustomMetrics({
    onetable: OneTableInstance,
    owner: 'my-service',
})

You can emit metrics via:

await metrics.emit('Acme/Metrics', 'launches', 10)

This will emit the launches metric in the Acme/Metrics namespace with the value of 10.

A metric can have dimensions that are unique metrics for a specific instance. For example, we may want to count the number of launches for a specific rocket.

await metrics.emit('Acme/Metrics', 'launches', 10, [{rocket: 'saturnV'}])

The metric will be emitted for each dimension provided. A dimension may have multiple properties.

If you want to emit a metric without dimensions, you can add {}. For example:

await metrics.emit('Acme/Metrics', 'launches', 10, [{}, {rocket: 'saturnV'}])
await metrics.emit('Acme/Metrics', 'launches', 10, [{}, {rocket: 'falcon9'}])

To query a metric, use the query method:

let results = await metrics.query('Acme/Metrics', 'speed', {rocket: 'saturnV'}, 'mth', 'max')

This will retrieve the 'speed' metric from the 'Acme/Metrics' namespace for the rocket == 'saturnV' dimension. The data points returned will be the maximum speed measured over the month during each interval. By default, the interval for the month span is 2 days.

This will return data like this:

{
    "namespace": "Acme/Metrics",
    "metric": "launches",
    "dimensions": {rocket: "saturnV"},
    "spans": [{
        "end": 946648800,
        "period": 300,
        "samples": 10,
        "points": [
            { "sum": 24000, "count": 19, "min": 1000, "max": 5000 },
            ...
        ]
    ]
}

If you want to query the results as a single value over the entire period (instead of as a set of data points), set the "accumulate" options to true.

let results = await metrics.query('Acme/Metrics', 'speed', {rocket: 'saturnV'}, 86400, 'max', {accumulate: true})

This will return a single maximum speed over the last day.

To obtain a list of metrics, use the getMetricList method:

let list: MetricList = await metrics.getMetricList()

This will return an array of available namespaces in list.namespaces.

To get a list of the metrics available, pass a metric as the first argument.

let list: MetricList = await metrics.getMetricList('Acme/Metrics')

This will return a list of metrics in list.metrics.

To get a list of the dimensions available for a metric, pass in a namespace and metric.

let list: MetricList = await metrics.getMetricList('Acme/Metrics', "speed")

This will return a list of dimensions in list.dimensions.

In all calls, the full list of namespaces will be returned regardless.

Limitations

While CustomMetrics does have options to buffer and coalesce metric updates, CustomMetrics can impose a meaningful DynamoDB write load if you are updating metrics extremely frequently. See Buffering below for mitigations.

Metric Schema

CustomMetrics are stored in a DynamoDB table using the following single-table schema.

const Schema = {
    format: 'onetable:1.1.0',
    version: '0.0.1',
    indexes: {primary: {hash: 'pk', sort: 'sk'}},
    models: {
        Metric: {
            pk: {type: 'string', value: 'metric#${version}#${owner}'},
            sk: {type: 'string', value: 'metric#${namespace}#${metric}#${dimensions}'},
            dimensions: {type: 'string', required: true, encode: ['sk', '#', '3']},
            expires: {type: 'date', ttl: true},
            metric: {type: 'string', required: true, encode: ['sk', '#', '2']},
            namespace: {type: 'string', required: true, encode: ['sk', '#', '1']},
            owner: {type: 'string', required: true, encode: ['pk', '#', '2']},
            version: {type: 'number', default: Version, encode: ['pk', '#', '1']},
            spans: {
                type: 'array',
                required: true,
                default: [],
                items: {
                    type: 'object',
                    default: {},
                    schema: {
                        // When the points will be full.
                        end: {type: 'number', required: true, map: 'se'}, 
                        period: {type: 'number', required: true, map: 'sp'},
                        samples: {type: 'number', required: true, map: 'ss'},
                        points: {
                            type: 'array',
                            required: true,
                            map: 'pt',
                            default: [],
                            items: {
                                type: 'object',
                                schema: {
                                    count: {type: 'number', required: true, map: 'c'},
                                    max: {type: 'number', map: 'x'},
                                    min: {type: 'number', map: 'm'},
                                    sum: {type: 'number', required: true, map: 's'},
                                    //  P-values and timestamp are not stored
                                    pvalues: {type: 'array', map: 'v'},
                                    timestamp: {type: 'number', map: 'e', },
                                },
                            },
                        },
                    },
                },
            },
            seq: {type: 'number', default: 0},
            _source: {type: 'string'}, // When set, bypass DynamoDB steams change detection
        }
    } as const,
    params: {
        partial: true,
        isoDates: true,
        nulls: false,
        timestamps: false,
        typeField: '_type',
    },
}

CustomMetrics Class API

The CustomMetrics class provides the public API for CustomMetrics and public properties.

CustomMetrics Constructor

const metrics = new CustomMetrics({
    onetable: db,
    owner: 'my-service',
})

The CustomMetrics constructor takes an options parameter and an optional context property.

The options parameter is of type object with the following properties:

For example:

const log = new CustomMetrics({
    onetable: onetable,
    owner: 'my-service',
    pResolution: 100,
    ttl: 6 * 24 * 86400,
})

CustomMetric spans define how each metric is processed and aged. The spans are an ordered list of metric interval periods. For example, the default spans calculate statistics for the periods: 5 minutes, 1 hour, 1 day, 1 week, 1 month and 1 year.

Via the spans constructor option you can provide an alternate list of spans for higher, lower or more granular resolution.

The default CustomMetrics spans are:

const DefaultSpans: SpanDef[] = [
    {period: 5 * 60, samples: 10}, //  5 mins, interval: 30 secs
    {period: 60 * 60, samples: 12}, //  1 hr, interval: 5 mins
    {period: 24 * 60 * 60, samples: 12}, //  24 hrs, interval: 2 hrs
    {period: 7 * 24 * 60 * 60, samples: 14}, //  7 days, interval: 1/2 day
    {period: 28 * 24 * 60 * 60, samples: 14}, //  28 days, interval: 2 days
    {period: 365 * 24 * 60 * 60, samples: 12}, //  1 year, interval: 1 month
]

The span period property is the number of seconds in that span. The samples property specifies the number of data points to be captured. The (period / points) value is the interval between computed data points. If you call emit() more frequently than this, CustomMetrics will agregate the extra values into the relevant span value.

Here is an example of a higher resolution set of spans that keep metric values for 1 minute, 5 minutes, 1 hour and 1 day.

const log = new CustomMetrics({
    onetable: onetable,
    owner: 'my-service',
    spans: [
        {period: 1 * 60, samples: 5}, //  interval: 5 secs
        {period: 5 * 60, samples: 10}, //  interval: 30 secs
        {period: 60 * 60, samples: 12}, //  interval: 5 mins
        {period: 24 * 60 * 60, samples: 12}, //  interval: 2 hrs
    ]
})

Buffering

If you have a metric that your app emits metrics at very high frequency, you may wish to optimize metrics by aggregating updates. CustomMetrics can aggregate metric updates by buffering emit calls. These are then persisted depending on your configured buffering policy.

For example:

await metrics.emit('Acme/Metrics', 'DataSent', 123, [], {
    buffer: {sum: 1024, count: 20, elapsed: 60}
})

This will buffer metric updates in-memory until the sum of buffered DataSent is greater than 1024, or there have been 20 calls to emit, or 60 seconds has elapsed, whichever is reached first. If elapsed is omitted, the default elapsed period is the period of your lowest span. CustomMetrics will regularly flush metrics as required and will save buffered metrics upon Lambda instance termination.

Buffered metrics may be less accurate than non-buffered metrics. Metrics may be retained in-memory for a period of time before being flushed to DynamoDB. If a Lambda instance is not required to service a request, any buffered metrics will remain in-memory until AWS terminates the Lambda -- whereupon the buffered values will be saved. This may mean a temporary loss of accuracy to querying entities.

Furthermore, if you have a very large number of metrics in one Lambda instance, it is possible that the Lambda instance may not be able to save all buffered metrics during the Lambda termination timeout. This can be somewhat mitigated by using shorter buffering criteria.

For these reasons, don't use buffered metrics if you require absolute precision. But if you have metrics where less than perfect accuracy is acceptable, then buffered metrics can give very large performance gains.

Methods

emit

Emit one or more metrics.

async emit(namespace: string, 
    metric: string, 
    value: number, 
    dimensions: MetricDimensionsList = [{}],
    options?: {
        buffer: {
            sum: number, 
            count: number, 
            elapsed: number,
        }
        timestamp?: number
    }): Promise<void>

This call will emit metrics for each of the specified dimensions using the supplied namespace and metric name. These will be combined with the CustomMetrics owner supplied via the constructor to scope the metric.

For example:

await metrics.emit('Acme/Metrics', 'launches', 10, 
    [{}, {rocket: 'saturnV'}, {mission: 'ISS-service'}])

This will create three metrics:

The buffer option can be provided to optimize metric load by aggregating calls to emit(). See Buffering for details.

query

Query a metric value.

async query(namespace: string, 
    metricName: string, 
    dimensions: MetricDimensions, 
    period: number, 
    statistic: string, 
    options: MetricQueryOptions,
    Promise<MetricQueryResult>

This will retrieve a metric value for a given namespace, metric name and set of dimensions.

The period argument selects the metric span name to query. For example: 3600 for one hour.

The statistic can be avg, max, min, sum, count or a P-value of the form pNN where NN is the P-value. For example: p95 would return the P-95 value. To get meaningful P-value statistics you must set the CustomMetrics pResolution parameter to the number of data points to keep for computing P-values. By default this resolution is zero, which means P-values are not computed. To enable, you should set this to at least 100.

getMetricList

Return a list of supported namespaces, metrics and dimensions.

async getMetricList(
    namespace: string | undefined, 
    metric: string | undefined, 
    options = {fields, limit}): Promise<MetricList>

This call will return a MetricList of the form:

type MetricList = {
    namespaces: string[]
    metrics?: string[]
    dimensions?: MetricDimensions[]
}

The list of namespaces will always be returned. If a namespace argument is provided, the list of metrics in that namespace will be returned. If a metric argument is provided, the list of dimensions for that metric will be returned.

References

• SenseDeep Blog
• SenseDeep Web Site
• SenseDeep Developer Studio

Participate

All feedback, discussion, contributions and bug reports are very welcome.

• discussions
• issues

SenseDeep

A great way to view CustomMetrics is with SenseDeep. You can create dashboards with graphs, gauges and numerical widgets to display, monitor and alert on your metrics.

Contact

You can contact me (Michael O'Brien) on Twitter at: @mobstream, and read my Blog.