Introduction

asm-pm-runner runs postman collections and export results to crs format for usage in Apica ASM. The tool is based on the standard Postman Newman utility. See Newman npm at. The main enhancement to Newman is support for decryption of data files and certificates. Encryption/decryption is based on the cryptify npm module.

Installation

npm install -g @apica-io/asm-pm-runner

Command line Options

Usage: asm-pm-runner [options] <collection>

Options:
  -dk, --decryptKey <decryptKey>      Decrypt key
  -ev, --envVars [envVars...]         Environment variables. Format name=value format
  -v, --verbose                       Print collection information on stdout (default: false)
  -r, --resultDir <dir>               The result directory
  -dd, --dataDir <dir>                The data directory for certificates and test data
  -l, --logLevel <logLevel>           Log level in log4js (default: "info")
  --sslClientCert <path>              Specify the path to a client certificate (PEM)
  --sslClientKey <path>               Specify the path to a client certificate private key
  --sslClientPassphrase <passphrase>  Specify the client certificate passphrase (for protected key)
  -V, --version                       output the version number
  -h, --help                          display help for command

Samples

You will find some runnable examples in installation the directory under samples . See NPM documentation about where you can find the global npm directory. The package.json file contains scripts for running the samples with correct parameter settings.

Example 1 - Running some http requests on http-bin.

$ asm-pm-runner samples/HTTP-Bin-Requests.postman_collection.json -v -ev username=foo password=bar

Example 2 - Testing certificate on https://badssl.com

This sample use encrypted certificates.

The command line

$ asm-pm-runner -l debug -dk  Encrypt.4.ssl --sslClientCert file.cert.pem --sslClientKey file.key.pem --dataDir samples/ec_badssl_certs -r results/ -v samples/BadSSL.postman_collection.json

The console output from the command

 [2021-08-18T10:08:21.470] [INFO] @apica-io/asm-pm-runner - @apica-io/asm-pm-runner (1.0.2) started
[2021-08-18T10:08:21.478] [DEBUG] @apica-io/asm-pm-runner - collection=samples/BadSSL.postman_collection.json
[2021-08-18T10:08:21.574] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem
[2021-08-18T10:08:21.580] [DEBUG] NewmanRunner - Decrypting /var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem
itemGroup BadSSL
item ClientBadSSL
request GET https://client.badssl.com/
response OK 200 OK 1891 { body: 687, header: 283, total: 970 }
timingPhases {
  secure_handshake_ms: 339.4973890000001,
  prepare: 73.57351800000015,
  socket_wait_ms: 59.02749899999981,
  dns_time_ms: 1147.0008660000003,
  tcp_connect_ms: 171.81406999999945,
  time_to_first_byte_ms: 152.74416699999983,
  download_time_ms: 19.97996400000011,
  process_time_ms: 1.6743750000005093,
  total_time_ms: 1965.3118480000003
}
assertion Check status code is 200 true
Collection BadSSL
run.stats {
  iterations: { total: 1, pending: 0, failed: 0 },
  items: { total: 1, pending: 0, failed: 0 },
  scripts: { total: 1, pending: 0, failed: 0 },
  prerequests: { total: 1, pending: 0, failed: 0 },
  requests: { total: 1, pending: 0, failed: 0 },
  tests: { total: 1, pending: 0, failed: 0 },
  assertions: { total: 1, pending: 0, failed: 0 },
  testScripts: { total: 1, pending: 0, failed: 0 },
  prerequestScripts: { total: 0, pending: 0, failed: 0 }
}
[2021-08-18T10:08:24.072] [DEBUG] NewmanRunner - Clean up temporary files [
  '/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.cert.pem',
  '/var/folders/g2/zb9q82pd18sd683qvmbyr0040000gn/T/31565_file.key.pem'
]
Result:  true Requests:1, tests:1, assertions:1
[2021-08-18T10:08:24.074] [DEBUG] NewmanRunner - Results written to /Users/janostgren/work/node/asm-pm-runner/results/BadSSL_res.json

The generated json report (crs format)

The json report report is stored in the results directory. The format is complaint with Check Result Service in Apica ASM. The name of result file will be {collection.name}_res.json. The environment variable APICA_JOB_ID can be used to set basename of the report file.

{
  "name": "BadSSL",
  "type": "Postman.Collection",
  "result_version": "1.0.2",
  "producer": "@apica-io/asm-pm-runner 1.0.2",
  "value": 1891,
  "unit": "ms",
  "returncode": 0,
  "success": true,
  "message": "Requests:1, tests:1, assertions:1",
  "duration_ms": 1891,
  "content_length": 687,
  "start_timestamp_ms": 1629274104025,
  "end_timestamp_ms": 1629274105916,
  "metrics": {
    "dns_time_ms": 1147,
    "download_time_ms": 20,
    "prepare": 74,
    "process_time_ms": 2,
    "secure_handshake_ms": 339,
    "socket_wait_ms": 59,
    "tcp_connect_ms": 172,
    "time_to_first_byte_ms": 153,
    "total_time_ms": 1965
  },
  "variables": [],
  "steps": [
    {
      "name": "BadSSL",
      "duration_ms": 1891,
      "content_length": 687,
      "start_timestamp_ms": 1629274104025,
      "end_timestamp_ms": 1629274105916,
      "metrics": {
        "dns_time_ms": 1147,
        "download_time_ms": 20,
        "prepare": 74,
        "process_time_ms": 2,
        "secure_handshake_ms": 339,
        "socket_wait_ms": 59,
        "tcp_connect_ms": 172,
        "time_to_first_byte_ms": 153,
        "total_time_ms": 1965
      },
      "requests": [
        {
          "name": "ClientBadSSL",
          "duration_ms": 1891,
          "content_length": 687,
          "start_timestamp_ms": 1629274104025,
          "end_timestamp_ms": 1629274105916,
          "metrics": {
            "secure_handshake_ms": 339,
            "prepare": 74,
            "socket_wait_ms": 59,
            "dns_time_ms": 1147,
            "tcp_connect_ms": 172,
            "time_to_first_byte_ms": 153,
            "download_time_ms": 20,
            "process_time_ms": 2,
            "total_time_ms": 1965
          },
          "url": "https://client.badssl.com/",
          "method": "GET",
          "status": 200,
          "status_text": "OK",
          "request_headers": {
            "User-Agent": "PostmanRuntime/7.28.2",
            "Accept": "*/*",
            "Cache-Control": "no-cache",
            "Postman-Token": "c0cfe5c6-d962-4dcc-a2bf-bef71a09f62c",
            "Host": "client.badssl.com",
            "Accept-Encoding": "gzip, deflate, br",
            "Connection": "keep-alive"
          },
          "headers": {
            "Server": "nginx/1.10.3 (Ubuntu)",
            "Date": "Wed, 18 Aug 2021 08:08:24 GMT",
            "Content-Type": "text/html",
            "Last-Modified": "Tue, 23 Feb 2021 21:28:41 GMT",
            "Transfer-Encoding": "chunked",
            "Connection": "keep-alive",
            "ETag": "W/\"60357389-2af\"",
            "Cache-Control": "no-store",
            "Content-Encoding": "gzip"
          }
        }
      ],
      "success": true,
      "assertions": [
        {
          "source": "ClientBadSSL",
          "description": "Check status code is 200",
          "status": "info",
          "expression": "Check status code is 200",
          "value": true
        }
      ]
    }
  ]
}