Extension APIs

Artillery is designed to be extensible and hackable to let users customize it if needed. Artillery is powered by Node.js, allowing you to leverage thousands of high-quality npm modules to do pretty much anything you may need to do.

This page covers:

• Different ways of extending and customizing Artillery
• How to track custom metrics from your tests
• How to customize VU behavior with custom JS code
• How Artillery plugins work

Extending VU behavior

Virtual user behavior can be customized via custom functions written in JS and attached to scenarios via extension points also known as "hooks". Different engines provide different extension points. Please refer to documentation of individual engines for details on what hooks they support, and what the APIs look like:

• HTTP
• Socket.IO
• WebSockets

Tracking custom metrics

You can track custom metrics through JS code. The following metric types are supported:

1. counters - a counter to keep track of certain events, like the number of times a VU completed a specific transaction.
2. histograms - a distribution (max, min, median, mean, p95, p99) of some measurements, like API response times or content length.
3. rates - the average number of times something happened in a time period, like the number of HTTP requests sent.

Custom metrics will get included in Artillery's output alongside standard metrics.

Custom metrics API

Custom metrics are sent through an EventEmitter object, available as an argument in hook functions and as a constructor parameter for plugins and engines.

• events.emit('counter', 'my_counter', 1) - Increments the value of a counter called my_counter by 1.
• events.emit('histogram', 'my_histogram', 42) - Records a measurement of 42 in a histogram called my_histogram.
• events.emit('rate', 'my_rate') - Record an instance of something happening, tracked as a rate metric named my_rate.

Example

Let's say we have an HTTP API with a route that creates a new pet object. We want to count how many pets we've created during the load test. Pets are also computationally expensive to create on the server, so we'd like to track the response time for the requests through the Server-Timing (opens in a new tab) response header.

The test script contains the following:

config:
  target: "https://pet-generator.test"
  processor: "./metrics.js"
  phases:
    - arrivalRate: 25
      duration: 300
 
scenarios:
  - afterResponse: "trackPets"
    flow:
      - post:
          url: "/pets"
          json:
            species: "pony"
            name: "Tiki"

The test script contains one scenario with an afterResponse hook. The hook executes the trackPets function located inside the metrics.js file:

module.exports = { trackPets };
 
function trackPets(req, res, context, events, done) {
  // After every response, increment the 'Pets created' counter by 1.
  events.emit("counter", "pets_created", 1);
 
  // Parse the 'server-timing' header and look for the 'pets' metric,
  // and add it to 'Pet creation latency' histogram.
  const latency = parseServerTimingLatency(
    res.headers["server-timing"],
    "pets"
  );
  events.emit("histogram", "pet_created_latency", latency);
 
  return done();
}
 
function parseServerTimingLatency(header, timingMetricName) {
  const serverTimings = header.split(",");
 
  for (let timing of serverTimings) {
    const timingDetails = timing.split(";");
    if (timingDetails[0] === timingMetricName) {
      return parseFloat(timingDetails[1].split("=")[1]);
    }
  }
}

The trackPets function collects two metrics after every response returned to a virtual user:

• A counter called pets_created, incremented by 1.
• A histogram called pet_created_latency, taken from the server-timing header.

When running the test script, Artillery will collect the custom metrics and display the metrics as part of the test results:

Plugins

Plugins provide a way to package up some functionality in an easy-to-use interface. Plugins in Artillery have full access to the test script and Artillery's internals, and it's possible to get very creative with what they can do.

Naming conventions

Artillery plugins are distributed as npm packages and follow the artillery-plugin-$NAME convention, e.g. artillery-plugin-publish-metrics (opens in a new tab).

Plugin lifecycle

1. Plugins are enabled with the config.plugins section of a test script, which makes Artillery try to load the plugin:

config:
  target: "http://svc-foo.prod.acme-corp.internal"
  plugins:
    my-custom-plugin:
      option1: some value
      option2: some other value

This example will make Artillery try to find and load the artillery-plugin-my-custom-plugin package.

1. The plugin package is expected to export a Plugin class. The constructor receives two arguments:

   1. script - The full test script definition, which your plugin code can modify. For example, a plugin may attach additional hook functions to scenario definitions, like the metrics-by-endpoint (opens in a new tab) plugin.
   2. events - An EventEmitter which receives events from Artillery and can also send custom metrics, like the publish-metrics (opens in a new tab) plugin.

2. A plugin may define a cleanup function, which takes a done callback. Artillery will call this function before exiting when a test finishes, providing an opportunity for any cleanup that a plugin needs to do. For example, you can use the function to flush buffered metrics to an external monitoring system.

Plugin events

Plugins can subscribe to the following events:

1. phaseStarted - A new arrival phase has started.
2. phaseCompleted - An arrival phase has finished.
3. stats - Provides metrics for the previous reporting period.
4. done - All VUs finished running their scenarios.

Attaching function objects to script from a Plugin

In some cases, you may need your plugin to attach javascript functions to the script, to be run by the Artillery engine (expect plugin works like this, for example). However, since Artillery v2 is multi-threaded, function objects cannot be sent from the main thread to worker threads. That means that plugins need to detect when they're running inside a worker thread.

If your plugin has this requirement, make sure to add code like this as early as possible in the Plugin (e.g. in the constructor):

  if (typeof process.env.LOCAL_WORKER_ID === 'undefined') {
    debug('Not running in a worker, exiting');
    return;
  }

Example

The artillery-plugin-hello-world (opens in a new tab) package contains an example plugin to learn about Artillery's plugin interface and can serve as a starting point to create your custom plugins.

Engines

Engines provide support for different actions inside virtual user scenarios. For example, the built-in HTTP engine provides support for get, post, put, and other HTTP actions.

An engine may package up a way to communicate with a particular type of server in Artillery, like HTTP, Socket.IO, or WebSockets. Such an engines may work on levels 4 or 7 of the OSI model (opens in a new tab).

Engines may also provide actions for a higher-level protocol level like an SQL engine or a AWS Kinesis (opens in a new tab) engine.

They may also be used to package up application-specific actions to make a particular application easier to test. For example an engine for a chat and collaboration platform like Slack or Discord may provide actions such as registerUser, loginAsUser, joinChannel, postMessage and so on.

Reporters

Custom reporters to send metrics and reports to external systems can get implemented as a plugin.

Please see the official publish-metrics (opens in a new tab) plugin for a complete example of a custom reporter.