Match Pattern

This is a general purpose validation tool for JSON objects. It includes facilities for deep matching, partial matching, unordered lists, and several advanced features for complex patterns. It also includes a variety of validation functions from the lodash-checkit module (a lodash extension mashup with checkit), and it allows for custom checking and mapping functions.

The primary goal of this and the supporting modules is to enable the highly flexible, expressive, and resilient feature testing of JSON based APIs.

Basic Usage

npm install lodash-match-pattern --save-dev

In your test file insert

var matchPattern = require('lodash-match-pattern');
var _ = matchPattern.getLodashModule(); // Use our lodash extensions (recommended)

// Example usage:

var testValue = {a: 1, b: 'abc'};

var successResult = matchPattern(testValue, {a: 1, b: _.isString});
// returns null for a successful match.

var failResult = matchPattern(testValue, {a: _.isString, b: 'abc'});
// returns "{a: 1} didn't match target {a: '_.isString'}"

Features Index

You probably won't need all of these features, but there's plenty of flexibility to to adapt to the details of your specific use cases. All of the examples below are illustrated in the examples/example1/features/basic.feature as cucumber-js tests.

1. Deep JSON matching
2. Matching property types
3. Partial objects
4. Partial, superset, and unordered arrays
5. Omitted items
6. Parametrized matchers
7. Transforms
8. Apply transform example
9. Map pattern transform example
10. Map values transform example
11. Composition and multiple transforms
12. Customization

Specification with JavaScript Objects or "Pattern Notation"

There are two similar ways to specify patterns to match. JavaScript objects are more convenient for mocha and JavaScript test runners that aren't multiline string friendly. "Pattern Notation" is more readable in cucumber tests and other environments that support multiline strings. Almost all patterns can be expressed in either form. In most cases below examples will be shown in both forms.

Deep JSON matching

Just for starters, suppose we have a "joeUser" object and want to validate its exact contents. Then matchPattern will do a deep match of the object and succeed as expected.

var matchPattern = require('lodash-match-pattern');
var joeUser = getJoeUser();

describe('basic match', function () {
  it('matches joeUser', function () {
    var matchResult = matchPattern(joeUser,
{
  id: 43,
  email: 'joe@matchapattern.org',
  website: 'http://matchapattern.org',
  firstName: 'Joe',
  lastName: 'Matcher',
  createDate: '2016-05-22T00:23:23.343Z',
  tvshows: [
    'Match Game',
    'Sopranos',
    'House of Cards'
  ],
  mother: {
    id: 23,
    email: 'mom@aol.com'
  },
  friends: [
    {id: 21, email: 'bob@mp.co', active: true},
    {id: 89, email: 'jerry@mp.co', active: false},
    {id: 14, email: 'dan@mp.co', active: true}
  ]
};
    if (matchResult) throw(new Error(matchResult));
  });
});

  Given I have joeUser
  Then joeUser matches the pattern
    """
{
  id: 43,
  email: 'joe@matchapattern.org',
  website: 'http://matchapattern.org',
  firstName: 'Joe',
  lastName: 'Matcher',
  createDate: '2016-05-22T00:23:23.343Z',
  tvshows: [
    'Match Game',
    'Sopranos',
    'House of Cards'
  ],
  mother: {
    id: 23,
    email: 'mom@aol.com'
  },
  friends: [
    {id: 21, email: 'bob@mp.co', active: true},
    {id: 89, email: 'jerry@mp.co', active: false},
    {id: 14, email: 'dan@mp.co', active: true}
  ]
});
    """

Notes

• In this case the JS Object and the Pattern Notation are visually identical. The only difference is the first is a JS object and the second is a string.
• For all the following examples we'll leave out the surrounding test boiler plate.
• For completeness the cucumber step definitions could be defined as:

// steps.js
var matchPattern = require('lodash-match-pattern');
module.exports = function () {
  var self = this;

  self.Given(/^I have joeUser$/, function () {
    self.user = {
      id: 43,
      email: 'joe@matchapattern.org',
      ...
    }
  });

  self.Then(
    /^joeUser matches the pattern$/,
    function (targetPattern) {
      var matchResult = matchPattern(self.user, targetPattern);
      if (matchResult) throw matchResult;
    }
  );
};

Unfortunately, deep matching of exact JSON patterns creates over-specified and brittle feature tests. In practice such deep matches are only useful in small isolated feature tests and occasional unit tests. Just for example, suppose you wanted to match the exact createDate of the above user. Then you might need to do some complex mocking of the database to spoof a testable exact value. But the good news is that we don't really care about the exact date, and we can trust that the database generated it correctly. All we really care about is that the date looks like a date. To solve this and other over-specification problems lodash-match-pattern enables a rich and extensible facility for data type checking.

Matching property types

The pattern below may look a little odd at first, but main idea is that there's a bucket full of _.isXxxx matchers available to check the property types. All you need to do is slug in the pattern matching function and that function will be applied to the corresponding candidate value.

{
  id: _.isInteger,
  email: _.isEmail,
  website: _.isUrl,
  firstName: /[A-Z][a-z]+/,
  lastName: _.isString,
  createDate: _.isDateString,
  tvshows: [
    _.isString,
    _.isString,
    _.isString
  ],
  mother: _.isObject,
  friends: _.isArray
}

Notes

• Again the two forms are visually identical. However there's one significant difference. For the JS Objects the matching functions (e.g _.isString) can be any function in scope. In contrast the corresponding Pattern Notation functions are required to be members of our lodash extension module and are required to begin with "is".

• The available matching functions are

  1. All isXxxx functions from lodash.
  2. All validation functions from checkit with is prepended.
  3. Case convention matchers constructed from lodash's ...Case functions.
  4. Any regular expression -- intepreted as /<regex>/.test(<testval>).
  5. isDateString, isSize, isOmitted
  6. Any isXxxx function you insert as a lodash mixin through customization.

To see the full list run this:

console.log(
  Object.keys(require('lodash-match-pattern').getLodashModule())
  .filter(function (fname) { return /^is[A-Z]/.test(fname) })
);

Partial objects

Most of the time ,feature tests are interested in how objects change, and we don't need be concerned with properties of an object that aren't involved in the change. In fact a principle of feature testing requires elimination of such incidental details. Matching only partial objects can create a huge simplification which focuses on the subject of the test. For example if we only wanted to test changing our user's email to say "billybob@duckduck.go" then we can simply match the pattern:

{
  id: 43,
  email: 'billybob@duckduck.go',
  '...': ''
}

{
  id: 43,
  email: 'billybob@duckduck.go',
  ...
}

The '...' object key indicates that only the specified keys are matched, and all others in joeUser are ignored.

Note: from here on all the examples will use partial matching, and all will successfully match "joeUser".

Partial, Superset, and Unordered Arrays

Similarly matching of partial arrays (as well as supersets and set equality) can be easily specified, but with a couple caveats:

1. The array entries must be numbers or strings, no nested objects or arrays.
2. The partial (and supersets) arrays are matched as sets -- no order assumed.

{
  tvshows: [
    'House of Cards',
    'Sopranos',
    '...'
  ],
  '...': ''
}

{
  tvshows: [
    'House of Cards',
    'Sopranos',
    ...
  ],
  ...
}

Note that the above specifies both a partial array (for joeUser.tvshows) and a partial object (for joeUser).

Supersets are similarly specified by '^^^'. This following says that joeUser.tvshows is a subset of the list in the pattern below:

{
  tvshows: [
    'House of Cards',
    'Match Game',
    'Sopranos',
    'Grey's Anatomy',
    '^^^'
  ],
  '...': ''
}

{
  tvshows: [
    'House of Cards',
    'Match Game',
    'Sopranos',
    'Grey's Anatomy',
    ^^^
  ],
  ...
}

Or to compare equality of arrays as sets by unordered membership, use "===":

{
  tvshows: [
    'House of Cards',
    'Match Game',
    'Sopranos',
    '==='
  ],
  '...': ''
}

{
  tvshows: [
    'House of Cards',
    'Match Game',
    'Sopranos',
    ===
  ],
  ...
}

Note that the JS Object specification adds the set matching symbols as extra array elements. If you actually need to literally match "...", "^^^", or "===" in an array see the customization example below.

Omitted items

Sometimes an important API requirement specifies fields that should not be present, such as a password. This can be validated with an explicit _.isOmitted check (an alias of _.isUndefined). Note that it works properly with partial objects.

{
  id: 43,
  password: _.isOmitted,
  '...': ''
}

{
  id: 43,
  password: _.isOmitted,
  ...
}

Parametrized matchers

Some of the matching functions take parameters. These can be specified with "|" separators at the end of the matching function.

{
  id: _.isBetween|42.9|43.1,
  tvshows: '_.isContainerFor|'House of Cards'',
  '...': ''
}

{
  id: _.isBetween|42.9|43.1,
  tvshows: _.isContainerFor|'House of Cards',
  ...
}

Transforms

Transforms modify the test data in some way before applying a match pattern. Transforms can be applied at any level of the match object and they may be composed. Use transforms sparingly since they tend to make the patterns less readable, and they could be a code smell of excessively complex tests. In many cases separate tests or custom matcher will be clearer.

Apply Transform Example

As a simple motivation consider matching a compound object such at the joeUser's friends list. We may not be able to guarantee order of items returned in the list, and probably don't care anyway. So explicitly matching the friends in a specific order will probably be an unreliable test. (The above "===" array set specifier only applies to arrays of primitives.) To fix this a <-.sortBy transform can be applied to force the test data into a specific order that can be reliably tested.

{
  friends: {
    '<-.sortBy|email': [
      {id: 21, email: 'bob@mp.co', active: true},
      {id: 14, email: 'dan@mp.co', active: true},
      {id: 89, email: 'jerry@mp.co', active: false}
    ]
  },
  '...': ''
}

{
  friends: {
    <-.sortBy|email: [
      {id: 21, email: 'bob@mp.co', active: true},
      {id: 14, email: 'dan@mp.co', active: true},
      {id: 89, email: 'jerry@mp.co', active: false}
    ]
  },
  ...
}

Any function in lodash-checkit is available for use in transforms, along with any function you add via customization. The functions are applied with the testValue as the function's first argument, and additional | separated arguments can be specified after the function name.

Important Note: The transform functions are applied to the test value, NOT the corresponding test pattern. In this example we are testing the joeUser.friends list. So this list is sorted by email and the resulting array is tested against the pattern specified in the above array pattern.

Map Pattern Transform Example

Suppose you just wanted to check that all of of joeUser's friends have emails ...@mp.co.

{
  friends: {
    '<=': { email: /@mp.co$/, '...': ''}
  },
  '...': ''
}

{
  friends: {
    <=: { email: /@mp.co$/, ...}
  },
  ...
}

The <= specifies that the pattern is applied to each of the entries of the joeUser.friends array. This is in contrast to the <- operator would specify that the pattern is matched against the array as a whole.

Map Values Transform Example

Suppose you wanted to check that joeUser's friends are in a "whitelist" of emails. Then you need to extract the emails, and since the whitelist check is case insensitive you need to compare them all in lower case. The following example expresses these requirements:

{
  friends: {
    '<=.get|email': {
      '<=.toLower': [
        'bob@mp.co',
        'jerry@mp.co',
        'dan@mp.co',
        'paul@mp.co',
        '^^^': ''
      ]
    }
  },
  '...': ''
}

{
  friends: {
    <=.get|email: {
      <=.toLower: [
        'bob@mp.co',
        'jerry@mp.co',
        'dan@mp.co',
        'paul@mp.co',
        ^^^
      ]
    }
  },
  ...
}

Here <=.get|email specifies that the _.get(..., 'email') is applied to each of the entries of the joeUser.friends array and creates a new array. In turn <=.toLower creates a mapped array with all emails in lower case. The result is then compared to the given whitelist.

Map transforms can be applied to objects as well as arrays. For arrays <=.lodashFunction uses _.map to apply _.lodashFunction to each array element. For objects _.mapValues is used instead.

Composition and Multiple Transforms

Transformations can be mixed and matched. Multiple transforms can also appear as keys in a single object. In that case they check the test value against all their respective pattern values. Notice, as suggested in the previous example, transform compositions are always applied to the test value from the outside to the inside where they result in the final pattern match.

In the following artificial example verifies that joeUser has "2" active friends, in 4 different ways.

{
  friends: {
    '<-.filter|active': {
      '<-.size': 2,
      '<-': _.isSize|2,
      '<-.isSize|2': true
    },
    '<=.get|active': {
      '<=.toNumber': {
        '<-.sum': 2
      }
    }
  },
  '...': ''
}

{
  friends: {
    <-.filter|active: {
      <-.size: 2,
      <-: _.isSize|2,
      <-.isSize|2: true
    },
    <=.get|active: {
      <=.toNumber: {
        <-.sum: 2
      }
    }
  },
  ...
}

Customization

In many cases, application of transforms will create unintuitive and hard to understand pattern specifications. Fortunately creating custom matchers and custom transforms is easily accomplished via lodash mixins. For our examples we've added two lodash mixins in our example code:

var matchPattern = require('lodash-match-pattern');
var _ = matchPattern.getLodashModule();

_.mixin({
  literalSetToken: function (elem) {
    if (elem === '...') return 'LITERAL...';
    if (elem === '^^^') return 'LITERAL^^^';
    if (elem === '===') return 'LITERAL===';
    return elem;
  },

  isActiveSize: function (array, size) {
    if (! _.isArray(array)) return false;
    var activeSize = _.size(_.filter(array, 'active'));
    return activeSize === size;
  }
});

Then we have yet another (but simpler) method for counting joeUser's active friends.

{
  friends: _.isActiveSize|2,
  ...
}

The custom literalSetToken transform can be used to enable literal pattern matching of "..." and "---" in arrays. So for example, suppose for some reason joeUser had this as his actual tvshows list:

  [
    "===",
    "Mannix",
    "Game of Thrones",
    "...",
    "^^^"
  ]

Then the following now has a successful pattern match:

{
  tvshows: {
    '<=.literalSetToken': [
      'LITERAL===',
      'Mannix',
      'Game of Thrones',
      'LITERAL...',
      'LITERAL^^^'
    ]
  },
  '...': ''
}

{
  tvshows: {
    <=.literalSetToken: [
      'LITERAL===',
      'Mannix',
      'Game of Thrones',
      'LITERAL...',
      'LITERAL^^^'
    ]
  },
  ...
}