Testing HTTP

Overview

This page covers HTTP testing functionality in Artillery provided by the built-in HTTP engine.

HTTP-specific configuration

HTTP-specific configuration options that may be set in the config section of a test script.

TLS/SSL

By default, Artillery will reject SSL certificates that it’s unable to validate, e.g. self-signed certificates. You may see errors such as UNABLE_TO_GET_ISSUER_CERT_LOCALLY, UNABLE_TO_VERIFY_LEAF_SIGNATURE, CERT_UNTRUSTED or one of the other validation error codes when that happens.

You can disable certificate validation with either of the following two options:

  1. Pass -k (or --insecure) flag to artillery run or artillery quick
  2. Set rejectUnauthorized: false in config.tls:
config:
  target: "https://myapp.staging:3002"
  tls:
    rejectUnauthorized: false
scenarios:
  - ...

Note: This option can be useful for testing in a development / staging environment, but should never be used when testing a production system.

Request timeout

If a response takes longer than 120 seconds Artillery will abort the request and report an ETIMEDOUT error.

To increase or decrease the default timeout set config.http.timeout to a number (in seconds).

config:
  target: "http://my.app"
  http:
    # Responses have to be sent within 10 seconds or the request will be aborted
    timeout: 10

Fixed connection pool

By default Artillery will open a new connection for each new virtual user. To open and re-use a fixed number of connections instead, set config.http.pool to a number:

config:
  target: "http://my.app"
  http:
    pool: 10 # All HTTP requests from all virtual users will be sent over the same 10 connections

This can be useful to emulate the conditions when the target would normally be behind a load-balancer and would have a fixed number of connections established at any given time.

Max sockets per virtual user

By default Artillery creates one TCP connection per virtual user. To allow for multiple sockets per virtual user (to mimic the behavior of a web browser for example), the config.http.maxSockets option may be set.

Note: this setting is per virtual user, not for the total number of sockets. To limit the total number of sockets, use the pool setting.

Scenario actions and configuration

This section describes functionality available in scenarios described in the scenarios section of a test script.

GET / POST / PUT / PATCH / DELETE requests

An HTTP request object may have the following attributes:

  • url - the request URL; it will be appended to the target but can be fully qualified also
  • json - a JSON object to be sent in the request body
  • body - arbitrary data to be sent in the request body
  • headers - a JSON object describing header key-value pairs
  • cookie - a JSON object describing cookie key-value pairs
  • capture - use this to capture values from the response body of a request and store those in variables

Example:

config:
  target: "https://example.com"
  phases:
    - duration: 10
      arrivalRate: 1
scenarios:
  - flow:
      - get:
          url: "/"
      - post:
          url: "/resource"
          json:
            hello: "world"

Logging

Debug messages can be logged with the log action:

config:
  target: "https://example.com"
  phases:
    - duration: 10
      arrivalRate: 1
scenarios:
  - flow:
      - log: "New virtual user running"
      - get:
          url: "/"
      - post:
          url: "/resource"
          json:
            hello: "world"

The string argument to log may include variables:

- log: "Current environment is set to: {{ $environment }}"

Setting Headers

Arbitrary headers may be sent with:

- get:
    url: "/test"
    headers:
      X-My-Header: "123"

Compressed Responses (gzip)

Set gzip to true to have Artillery add an Accept-Encoding header to the request, and decode compressed responses (e.g. encoded with gzip).

- post:
    url: "/test"
    json:
      foo: bar
    gzip: true

Basic Auth

- get:
    url: "/protected/resource"
    auth:
      user: myusername
      pass: mypassword

Query Strings

Query strings can be appended directly to the url or set with qs:

- get:
    url: "/products"
    qs:
      search_keyword: "coffee"
      page_size: 25

Redirects

Artillery follows redirects by default. To stop Artillery from following redirects, set followRedirect property on a request to false.

- get:
    url: "/test"
    followRedirect: false

Forms

URL-encoded forms (application/x-www-form-urlencoded)

Use the form attribute to send an URL-encoded form.

- post:
    url: "/upload"
    form:
      name: "Homer Simpson"
      favorite_food: "donuts"

Multipart forms (multipart/form-data)

Use the formData attribute to send a multipart/form-data form (forms containing files, non-ASCII data, and binary data).

- post:
    url: "/upload"
    formData:
      name: "Homer Simpson"
      favorite_food: "donuts"

To attach binary data a custom JS function can be used.

First-class file upload support is also provided by Artillery Pro for teams requiring extensive file uploading functionality in their tests.

Extracting and reusing parts of a response (request chaining)

You can parse responses and reuse those values in subsequent requests with a capture attribute.

Syntax

To tell Artillery to parse a response, add a capture attribute to any request spec like so:

- get:
    url: "/"
    capture:
      json: "$.id"
      as: "id"

The capture element must always have an as attribute which names the value for use in subsequent requests, and one of:

  • a json attribute containing a JSONPath expression
  • an xpath attribute containing an XPath expression
  • a regexp attribute containing a regular expression (a string that gets passed to a RegExp constructor. A specific capturing group to return may be set with the group attribute (set to an integer index of the group in the regular expression). Flags for the regular expression may be set with the flags attribute.
  • a header attribute containing the name of the response header whose value you want to capture
  • a selector attribute containing a Cheerio element selector; an attr attribute containing the name of the attribute whose value we want to be returned, and optionally an index attribute may be set to a number, "random" or "last" to grab an element matching the selector at the specified index, at random, or the last matching one. If an index not specified, the first matching element is returned.

Optionally, a transform attribute may be specified too, which should be a snippet of JS code (as a string) transforming the value after it has been extracted from the response:

- get:
    url: "/journeys"
    capture:
      xpath: "(//Journey)[1]/JourneyId/text()"
      transform: "this.JourneyId.toUpperCase()"
      as: "JourneyId"

Where this refers to the context of the virtual user running the scenario, i.e. an object containing all currently defined variables, including the one that has just been extracted from the response.

Turn off strict capture

By default, captures are strict, i.e. if a capture rule fails because nothing matches, any subsequent requests in the scenario will not run, and that virtual user will stop. This behavior is the default since most of the time subsequent requests depend on captured values and would fail when one is not available.

In some cases it may be useful to turn this behavior off. To make virtual users continue with running requests even after a failed capture, set strict to false:

- get:
    url: "/"
    capture:
      json: "$.id"
      as: "id"
    strict: false # We don't mind if id can't be captured and the next requests 404s
- get:
    url: "/things/{{ id }}"

Capturing multiple values

Multiple values can be captured with an array of capture specs, e.g.:

- get:
    url: "/journeys"
    capture:
      - xpath: "(//Journey)[1]/JourneyId/text()"
        transform: "this.JourneyId.toUpperCase()"
        as: "JourneyId"
      - header: "x-my-custom-header"
        as: "headerValue"

Examples

In the following example, we POST to /pets to create a new resource, capture part of the response (the id of the new resource) and store it in the variable id. We then use that value in the subsequent request to load the resource and to check to see if the resource we get back looks right.

  - post:
      url: "/pets"
      json:
        name: "Mali"
        species: "dog"
      capture:
        json: "$.id"
        as: "id"
  - get:
      url: "/pets/{{ id }}"
      match:
        json: "$.name"
        value: "{{ name }}"

In the following example, we grab a matching a element at random and then use the value of its href attribute in another request:

  - get:
      url: "/some/page"
      capture:
        - selector: "a[class^=productLink]"
          index: "random"
          attr: "href"
          as: "productUrl"
  - get:
      url: "{{ productUrl }}"

Cookies

Cookies are remembered and re-used by individual virtual users. Custom cookies can be specified with cookie attribute in individual requests.

  - get:
      url: "/pets/"
      cookie:
        saved: "tapir,sloth"

Pausing execution with think

To pause the virtual user for N seconds, use a think action:

  - post:
      url: "/pets"
      json:
        name: "Mali"
        species: "dog"
      capture:
        json: "$.id"
        as: "id"
  #
  # wait for 5 seconds:
  #
  - think: 5
  - get:
      url: "/pets/{{ id }}"
      match:
        json: "$.name"
        value: "{{ name }}"

Conditional Requests

An ifTrue attribute may be used to only execute a request in a flow if a condition is met. ifTrue may take two forms:

  1. Set to a name of a scenario variable. If the variable is set, the request will be executed.
  2. Set to a simple conditional expression, which may refer to any of the scenario variables and use one or more of:
    1. A numeric operation with +, -, *, /, % (modulo), or ^ (power)
    2. A comparison with ==, <, >, <=, >=
    3. Boolean operation with or, and, or not

Examples

Make a GET request only if the “keyword” scenario variable is set to a value:

- get:
    url: "/search?keyword={{ keyword }}"
    ifTrue: "keyword"

Make a GET request only if the pageNumber scenario variable is < 10:

- get:
    url: "/pages/{{ pageNumber }}"
    ifTrue: "pageNumber < 10"

Looping through a number of requests

You can use the loop construct to loop through a number of requests in a scenario. For example, each virtual user will send 100 GET requests to / with this scenario:

  config:
    # ... config here ...
  scenarios:
    - flow:
        - loop:
            - get:
                url: "/"
          count: 100

If count is omitted, the loop will run indefinitely.

loop is an array - any number of requests can be specified. Variables, cookie and response parsing will work as expected.

The current step of the loop is available inside a loop through the $loopCount variable (for example going from 1 too 100 in the example above).

Looping through an array

Looping through an array can be done by setting the over property to a literal array or the name of the variable containing an array of values.

In the following example 3 requests would be made, one for each product ID:

- loop:
    - get:
         url: "/products/{{ $loopElement }}"
   over:
     - id123
     - id456
     - id789

The product IDs could also be defined in the config section and used with a loop:

config:
  target: "https://my.app.local"
  phases:
    - duration: 600
      arrivalRate: 10
  variables:
    productIds:
      - ["id1", "id2", "id3"]
      - ["id4", "id5", "id6"]
scenarios:
  - flow:
      - loop:
          - get:
              url: "/products/{{ $loopElement }}"
        over: productIds

In this example each virtual user will make 3 requests to:

  • /products/id1, /products/id2 and /products/id3, or
  • /products/id4, /products/id5 and /products/id6

Looping with custom logic

Let’s say we want to poll an endpoint until it returns a JSON response with the top-level status attribute set to "ready":

- loop:
    - think: 5
    - get:
        url: "/some/endpoint"
        capture:
          - json: $.status
            as: "status"
  whileTrue: "myFunction"
function myFunction(context, next) {
  const continueLooping = context.vars.status !== 'ready';
  return next(continueLooping); // call back with true to loop again
}

NOTE: whileTrue true takes precendence over count and over attributes if either of those is specified.

Writing custom logic in Javascript

The HTTP engine has support for “hooks”, which allow for custom JS functions to be called at certain points during the execution of a scenario.

  • beforeScenario and afterScenario - called before/after a virtual user executes a scenario
  • beforeRequest - called before a request is sent; request parameters (URL, cookies, headers, body etc) can be customized here
  • afterResponse - called after a response has been received; the response can be inspected and custom variables can be set here
  • function which can be inserted as a step at any point in a scenario

JS functions may also be run at any point in a scenario with the function action.

Loading custom JS code

To tell Artillery to load your custom code, set config.processor to path to your JS file:

config:
  target: "https://my.app.dev"
  phases:
    -
      duration: 300
      arrivalRate: 1
  processor: "./my-functions.js"
scenarios:
  - # ... scenarios definitions here ...

The JS file is expected to be a standard Node.js module:

  //
  // my-functions.js
  //
  module.exports = {
    setJSONBody: setJSONBody,
    logHeaders: logHeaders
  }

  function setJSONBody(requestParams, context, ee, next) {
    return next(); // MUST be called for the scenario to continue
  }

  function logHeaders(requestParams, response, context, ee, next) {
    console.log(response.headers);
    return next(); // MUST be called for the scenario to continue
  }

Specifying a function to run

beforeRequest and afterResponse hooks can be set in a request spec like this:

  # ... a request in a scenario definition:
  - post:
      url: "/some/route"
      beforeRequest: "setJSONBody"
      afterResponse: "logHeaders"

This tells Artillery to run the setJSONBody function before the request is made, and to run the logHeaders function after the response has been received.

Specifying multiple functions

An array of function names can be specified too, in which case the functions will be run one after another.

Setting scenario-level hooks

Similarly, a scenario definition can have a beforeScenario/afterScenario attribute, which will make the functions specified run for every request in the scenario. The function signature is the same as for function hooks.

Function steps in a flow

A function may be run at any point in a scenario with a function action:

scenarios:
  - flow:
    # Call setupSomeData
    - function: "setupSomeData"
    - get:
        url: "/some/url?q={{ query }}"

Functions invoked with function action have full access to the virtual user’s context:

function setupSomeData(context, events, done) {
  context.vars['query'] = 'foo'; // set the "query" variable for the virtual user
  return done();)
}

Function signatures

beforeRequest

A function invoked in a beforeRequest hook should have the following signature:

function myBeforeRequestHandler(requestParams, context, ee, next) {
}

Where:

  • requestParams is an object given to the Request library. Use this parameter to customize what is sent in the request (headers, body, cookies etc)
  • context is the virtual user’s context, context.vars is a dictionary containing all defined variables
  • ee is an event emitter that can be used to communicate with Artillery
  • next is the callback which must be called for the scenario to continue; it takes no arguments

afterResponse

A function invoked in an afterResponse hook should have the following signature:

function myAfterResponseHandler(requestParams, response, context, ee, next) {
}

Where:

  • requestParams is an object given to the Request library. Use this parameter to customize what is sent in the request (headers, body, cookies etc)
  • response is likewise the response object from the Request library. This object contains response headers, body etc.
  • context is the virtual user’s context, context.vars is a dictionary containing all defined variables
  • ee is an event emitter that can be used to communicate with Artillery
  • next is the callback which must be called for the scenario to continue; it takes no arguments

Functions invoked with a function action, beforeScenario and afterScenario

function myFunction(context, ee, next) {
}

Where:

  • context is the virtual user’s context, context.vars is a dictionary containing all defined variables
  • ee is an event emitter that can be used to communicate with Artillery
  • next is the callback which must be called for the scenario to continue; it takes no arguments