Script Reference

An Artillery test script is composed of two sections: config and scenarios.

The config section defines the target (the hostname or IP address of the system under test), the load progression, and protocol-specific settings such as HTTP response timeouts or Socket.io transport options.

The scenarios section contains definitions for one or more scenarios for the virtual users that Artillery will create.

The config Section

Using dynamic values in config

Values can be set dynamically via environment variables which are available under $processEnvironment template variable.

For example, to set a default HTTP header for all requests via the SERVICE_API_KEY environment variable, your test definition would look like this:

config:
  target: https://my.microservice.internal
  phases:
    - duration: 600
      arrivalRate: 10
  defaults:
    headers:
      x-api-key: "{{ $processEnvironment.SERVICE_API_KEY }}"
scenarios:
  - flow:
      - get:
          url: "/"

Which would allow for the API key to be left out of the source code and provided on-the-fly with:

export SERVICE_API_KEY="012345-my-api-key"
artillery run my-test.yaml

Load Phases

A load phase defines how many new virtual users will be generated in a time period. For example, a typical performance test will have a gentle warm up phase, followed by a ramp up phase which is then followed by a maximum load for a duration of time.

config.phases is an array of phase definitions that Artillery goes through sequentially. Four kinds of phases are supported:

Examples

Create 50 virtual users every second for 5 minutes:

Constant arrival rate
config:
  target: "https://staging.example.com"
  phases:
    - duration: 300
      arrivalRate: 50
A ramp up followed by an constant arrival rate

Ramp up arrival rate from 10 to 50 over 2 minutes, followed by 10 minutes at 50 arrivals per second.

config:
  target: "https://staging.example.com"
  phases:
    - duration: 120
      arrivalRate: 10
      rampTo: 50
      name: "Warm up the application"
    - duration: 600
      arrivalRate: 50
      name: "Sustained max load"

(The name attributes are optional. They are useful to help identify load phases in the reports generated by Artillery.)

Fixed count of arrivals

Create 20 virtual users in 60 seconds (approximately one every 3 seconds):

config:
  target: "https://staging.example.com"
  phases:
    - duration: 60
      arrivalCount: 20
A do-nothing pause phase
config:
  target: "https://staging.example.com"
  phases:
    - pause: 60

How do ramps work?

Think of a rampTo as a shortcut for manually writing out a sequence of arrival phases, e.g. the following ramp:

phases:
  - duration: 100
    arrivalRate: 0
    rampTo: 50

is equivalent to:

phases:
  -
    arrivalRate: 0
    duration: 1.96
  -
    arrivalRate: 1
    duration: 1.96
  -
    arrivalRate: 2
    duration: 1.96
  -
     ... etc ...
  -
    arrivalRate: 50
    duration: 2

Environments

Typically we'd want to re-use a load testing script across multiple enviroments (e.g. dev, staging, and production) with minor tweaks. That's what config.environments is for: a number of named environments can be defined with environment-specific configuration.

For example, a typical use-case is to define multiple targets with different load phase definitions for each of those:

config:
  target: "http://wontresolve.local:3003"
  phases:
    - duration: 10
      arrivalRate: 1
  environments:
    production:
      target: "http://wontresolve.prod:44321"
      phases:
        - duration: 120
          arrivalRate: 10
    staging:
      target: "http://127.0.0.1:3003"
      phases:
        - duration: 1200
          arrivalRate: 20
scenarios:
  - ...

Choose an environment on the command line with the -e flag; e.g. artillery run -e staging my-script.yml.

The $environment variable

The name of the current environment (if set) is available in the $environment variable.

Example - printing the name of the current environment from a scenario:

config:
  #
  # config here
  #
scenarios:
  - flow:
    - log: "Current environment is set to: {{ $environment }}"

Payload Files

It can often be useful to be able to inject data from external files into your test scenarios. For example, you might have a list of usernames and passwords that you want to use to test the auth endpoint in your API.

Payload files are in the CSV format and Artillery allows you to map each of the rows to a variable name that can be used in scenario definitions. For example:

  config:
    payload:
      # path is relative to the location of the test script
      path: "users.csv"
      fields:
        - "username"
        - "password"
  scenarios:
    - flow:
        - post:
            url: "/auth"
            json:
              username: "{{ username }}"
              password: "{{ password }}"

We tell Artillery to load users.csv file and make variables username and password available in scenarios containing values from one of the rows in the CSV file

To import multiple CSV files "payload" may also be an an array:

payload:
  -
    path: "./pets.csv"
    fields:
      - "species"
      - "name"
  -
    path: "./urls.csv"
    fields:
      - "url"

It's also possible to load a different CSV file depending on the environment you set with the --environment flag. For example, you may want to load a different set of usernames/passwords to use with an authentication endpoint when running the same test in different environments:

payload:
  - path: "{{ $environment }}-logins.csv"
    fields:
      - "username"
      - "password"

Options

Example

config:
    payload:
      path: "users.csv"
      fields:
        - "username"
        - "password"
      order: sequence
      skipHeader: true
  scenarios:
    - # ... the rest of the script

Inline variables

Variables can defined in the config.variables section of a script and used in subsequent request templates.

  config:
    target: "http://app01.local.dev"
    phases:
      -
        duration: 300
        arrivalRate: 25
    variables:
      postcode:
        - "SE1"
        - "EC1"
        - "E8"
        - "WH9"
      id:
        - "8731"
        - "9965"
        - "2806"

The variables can then be used in templates as normal. For example: {{ id }} and {{ postcode }}.

Setting success conditions with ensure

When running Artillery in CI/CD pipelines, it can be useful to have Artillery exit with a non-zero code when a condition is not met.

Latency

To check that the aggregate p95 latency is 200ms or less, and make Artillery exit with a non-zero exit code if it's over 200, add the following configuration to your script:

config:
  ensure:
    p95: 200

min, max, median, p95, and p99 may be set.

Error rate

The error rate is defined as the ratio of virtual users that didn't complete their scenarios successfully to the total number of virtual users created during the test.

To make Artillery exit with a non-zero if the total error rate exceeded 1%:

config:
  ensure:
    maxErrorRate: 1

scenarios

The scenarios section is where one or more virtual user scenarios are defined.

A scenario is a sequence of steps that will be run sequentially which represents a typical sequence of requests or messages sent by a user of an application.

A scenario definiton is an object which must contain a flow attribute and may contain a number of other attributes.

Some Artillery engines will also support other scenario attributes, e.g. the HTTP engine allows for scenario-level hooks to be defined.

Scenario weights

Weights allow you to specify that some scenarios should be picked more often than others. If you have three scenarios with weights 1, 2, and 5, the scenario with the weight of 2 is twice as likely to be picked as the one with the weight of 1, and 2.5 times less likely than the one with weight of 5. Or in terms of probabilities:

Weights are optional, and if not specified are set to 1 (i.e. each scenario is equally likely to be picked).