JSPM

What is dynamodb-wrapper?

• Enhanced AWS SDK: Public interface closely resembles the AWS SDK, making it easier to learn and use.
• Bulk I/O: Easily read, write or delete entire collections of items in DynamoDB with a single API call.
• Events: Add event hooks to be notified of important events, such as whenever read/write capacity is consumed, or requests are retried due to throttling.
• Table prefixes: DynamoDBWrapper can add a table name prefix in requests and remove it in responses. This is helpful if you have multiple environments within the same AWS Account and region.

Installing

npm install dynamodb-wrapper

Usage

Setup

Construct the DynamoDBWrapper class

var AWS = require('aws-sdk');
var DynamoDBWrapper = require('dynamodb-wrapper');

var dynamoDB = new AWS.DynamoDB({
    // optionally disable AWS retry logic - reasoning explained below
    maxRetries: 0
});

// see the Configuration section of the README for more options
var dynamoDBWrapper = new DynamoDBWrapper(dynamoDB, {
    // optionally enable DynamoDBWrapper retry logic
    maxRetries: 6,
    retryDelayOptions: {
        base: 100
    }
});

(Optional) If you use DynamoDBWrapper retry logic instead of AWS retry logic, you gain the following benefits:

1. Improved batch processing: DynamoDBWrapper will automatically retry any UnprocessedItems in your batchWriteItem requests.
2. You can add a retry event listener to be notified when requests are throttled. In your application, you can log these events, or even respond by increasing provisioned throughput on the affected table.
3. DynamoDBWrapper's retryDelayOptions actually work as documented (this functionality doesn't work in the AWS JavaScript SDK yet, but there's an open ticket for this feature request).

dynamoDBWrapper.events.on('retry', function (e) {
    console.log(
        'An API call to DynamoDB.' + e.method + '() acting on table ' +
        e.tableName + ' was throttled. Retry attempt #' + e.retryCount +
        ' will occur after a delay of ' + e.retryDelayMs + 'ms.'
    );
});

// An API call to DynamoDB.batchWriteItem() acting on table MyTable
// was throttled. Retry attempt #3 will occur after a delay of 800ms.

(Optional) If you use the ReturnConsumedCapacity property in your AWS requests, the consumedCapacity event listener can notify you whenever read/write capacity is consumed.

dynamoDBWrapper.events.on('consumedCapacity', function (e) {
    console.log(
        'An API call to DynamoDB.' + e.method + '() consumed ' +
        e.capacityType, JSON.stringify(e.consumedCapacity, null, 2)
    );
});

// An API call to DynamoDB.batchWriteItem() consumed WriteCapacityUnits
// [
//   {
//     "TableName": "MyTable",
//     "CapacityUnits": 20
//   }
// ]

Example: Bulk Read

Read large collections of data from a DynamoDB table with a single API call. Multiple pages of data are aggregated and returned in a single response.

@see http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html

// params for Query - same format as in the AWS SDK
var sampleQueryParams = {
    TableName: 'MyTable',
    KeyConditionExpression: 'MyPartitionKey = :pk',
    ExpressionAttributeValues: {
        ':pk': {
            N: '1'
        }
    }
};

// fetches all pages of data from DynamoDB
// promise resolves with the aggregation of all pages,
// or rejects immediately if an error occurs

dynamoDBWrapper.query(sampleQueryParams)
    .then(function (response) {
        console.log(response.Items);
    })
    .catch(function (err) {
        console.error(err);
    });

Example: Bulk Write/Delete

Insert or delete large collections of items in a DynamoDB table with a single API call. DynamoDBWrapper batches your requests and aggregates the results into a single response. Use configuration values to fine tune throughput consumption for your use case.

@see http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html

// params for BatchWriteItem - same format as in the AWS SDK
var sampleParams = {
    RequestItems: {
        MyTable: [
            {
                PutRequest: {
                    Item: {
                        MyPartitionKey: { N: '1' }
                    }
                }
            },
            {
                DeleteRequest: {
                    Key: {
                        MyPartitionKey: { N: '2' }
                    }
                }
            },
            // this array can have thousands of items ...
        ]
    }
};

// performs all Puts/Deletes in the array
// promise resolves when all items are written successfully,
// or rejects immediately if an error occurs

dynamoDBWrapper.batchWriteItem(sampleParams, {
    // use configuration to control and optimize throughput consumption
    partitionStrategy: 'EvenlyDistributedGroupWCU',
    targetGroupWCU: 50,
    groupDelayMs: 1000
})
    .then(function (response) {
        console.log(response);
    })
    .catch(function (err) {
        console.error(err);
    });

Configuration

This section is copied from the IDynamoDBWrapperOptions interface in the index.d.ts file.

The DynamoDBWrapper constructor accepts an optional configuration object with the following properties:

• tableNamePrefix (string) - A prefix to add to all requests and remove from all responses.
• groupDelayMs (number) - The delay (in millseconds) between individual requests made by query(), scan(), and batchWriteItem(). Defaults to 100 ms.
• maxRetries (number) - The maximum amount of retries to attempt with a request. Note: this property is identical to the one described in the AWS documentation.
• retryDelayOptions (object) - A set of options to configure the retry delay on retryable errors. Note: this property is identical to the one described in the AWS documentation. Currently supported options are:
  • base (number) - The base number of milliseconds to use in the exponential backoff for operation retries. Defaults to 100 ms.
  • customBackoff (Function) - A custom function that accepts a retry count and returns the amount of time to delay in milliseconds. The base option will be ignored if this option is supplied.

API

The DynamoDBWrapper class supports a Promise-based API with the following methods. These are wrappers around the AWS SDK method of the same name. Please refer to the AWS API documentation and JavaScript SDK documentation for more details:

The following methods are passed straight through to the AWS JavaScript SDK:

• createTable(params)
• updateTable(params)
• describeTable(params)
• deleteTable(params)
• getItem(params)
• updateItem(params)
• putItem(params)
• deleteItem(params)

Enhanced API methods

The following API methods have enhanced behavior to support bulk I/O:

• query(params, options) - Fetches all pages of data from a DynamoDB query, making multiple requests and aggregating responses when necessary.
  • options.groupDelayMs (number) - the delay between individual requests. Overrides the configuration property of the same name in the constructor. Defaults to 100 ms.
• scan(params, options) - Fetches all pages of data from a DynamoDB scan, making multiple requests and aggregating responses when necessary.
  • options.groupDelayMs (number) - the delay between individual requests. Overrides the configuration property of the same name in the constructor. Defaults to 100 ms.
• batchWriteItem(params, options) - Writes or deletes large collections of items in a single DynamoDB table, batching items and making multiple requests when necessary.
  • options.groupDelayMs (number) - the delay between individual requests. Overrides the configuration property of the same name in the constructor. Defaults to 100 ms.
  • options.partitionStrategy (string) - strategy to use when partitioning the write requests array:
    • 'EqualItemCount' - creates groups with an equal number of items.
    • 'EvenlyDistributedGroupWCU' - creates groups with equalized total WCU, allowing for variable item counts.
  • options.targetItemCount (number) - the number of items to put in each group when using the EqualItemCount partition strategy.
  • options.targetGroupWCU (number) - the size threshold (in WriteCapacityUnits) of each group when using the EvenlyDistributedGroupWCU partition strategy.

Roadmap

• BatchGetItem: Add support for BatchGetItem as part of the "Bulk Read" feature set.
• Streams: Add method signatures that return Streams (instead of Promises), allowing for better integration ecosystems such as gulp