Extension APIs


What you’ll learn

  • Different ways of extending and customizing Artillery
  • How to track custom metrics from your tests
  • How to customize VU behavior with custom JS code
  • How Artillery plugins work

Artillery is designed to be extensible and hackable to let users customize it if needed. Artillery is powered by Node.js which means that thousands of high-quality npm modules can be leveraged to do pretty much anything you may need to do.

Extending VU behavior

Virtual user behavior can be customized via custom functions written in JS and attached to scenarios via extension points also known as “hooks”. Different engines provide different extension points, e.g. the HTTP engine supports the following hooks:

  • beforeRequest - run a function before a request is sent
  • afterResponse - run a function after a response has been received
  • function - run a function as a step anywhere in the scenario

Please refer to documentation of individual engines for details on what hooks they support, and what the APIs look like:

Tracking custom metrics

Custom metrics may be tracked from custom JS code. Two metric types are supported:

  1. counters - a count of some things that happened, e.g. the number of times VUs completed a certain transaction
  2. histograms - a distribution (max, min, p95, p99) of some measurement, e.g. API response time

Custom metrics are included in Artillery’s output alongisde standard metrics.

API

Custom metrics are sent through an EventEmitter object, which is available as an argument in hook functions, and as a constructor parameter for plugins and engines.

  • events.emit('counter', 'my_counter', 1) - increment the value of a counter called my_counter by 1
  • events.emit('histogram, 'my_histogram', 42) - record a measurement of 42 in a histogram called my_histogram

Examples

HTTP engine

Imagine, we have the following request in a scenario which creates a new pet object. We’d like to count how many pony objects we’ve created. Ponies are also computationally-expensive to create, so we’d like to track response time for pony-related requests specifically:

- post:
url: "/pets"
json:
species: "pony"
name: "Tiki"
afterResponse: "trackPonies"
//
// "events" is the EventEmitter we can use to create custom metrics.
// We're going to track a counter and a histogram.
//
function trackPonies(req, res, context, events, done) {
events.emit('counter', 'ponies_created', 1);
// parse Server-Timing header which provies timing information from the backend:
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing
const serverTiming = parseInt(res.headers['server-timing'].split('=')[1]);
events.emit('histogram', 'pony_latency', serverTiming);
return done();
}

Plugins

Plugins provide a way to package up some functionality in an easy to use interface. Plugins in Artillery have full access to the test script and Artillery’s internals, and it’s possible to get very creative with what they can do.

Plugin conventions

Artillery plugins are distributed as npm packages and follow the artillery-plugin-$NAME convention, e.g. artillery-plugin-publish-metrics.

Plugin lifecycle

  1. Plugins are enabled via config.plugins section of a test script, which makes Artillery try to load the plugin. For example:
    config:
    target: "http://svc-foo.prod.acme-corp.internal"
    plugins:
    my-custom-plugin:
    option1: some value
    option2: some other value
    This config will make Artillery try to find & load artillery-plugin-my-custom-plugin package.
  2. The plugin package is expected to export a Plugin class. The constructor receives two arguments:
    1. script - the entire test script definition, which may be modified by plugin code. For example: a plugin may attach additional hook functions to scenario definitions, like the metrics-by-endpoint plugin.
    2. events - an EventEmitter which receives events from Artillery, and may be used to send custom metrics, like the publish-metrics plugin.
  3. A plugin may define a cleanup function, which takes a done callback. Artillery will call this function when a test finishes before exiting, providing an opportunity for any cleanup that a plugin needs to do, like flushing buffered metrics to an external monitoring system for example.

Plugin events

Plugins can subscribe to the following events:

  1. phaseStarted - a new arrival phase has started
  2. phaseCompleted - an arrival phase has finished
  3. stats - provides metrics for the previous reporting period
  4. done - all VUs are finished

Engines

An engine packages up a way to communicate with a particular type of server in Artillery, like HTTP, Socket.io or WebSockets. Engines may work on levels 4 or 7 of the OSI model or on a higher-level protocol level, e.g. an SQL engine, or a AWS Kinesis engine.

We’re working on expanding this section in the documentation. In the meantime, the AWS Kinesis engine is a good place to start if you’re looking to write your own engine.

Reporters

Custom reporters to send metrics and reports to external systems may be implemented as plugins. Please see the official publish-metrics plugin for a complete example.