CloudWatch

Added inv2.0.0-28

This reporter supports sending metrics and traces to CloudWatch (opens in a new tab).

In order to enable the Cloudwatch reporter set type to cloudwatch, and region to the AWS region that you want metrics to be sent to:

config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: eu-west-1

Important: AWS credentials have to be present in the environment and have sufficient IAM permissions to send metrics to Cloudwatch and/or traces to AWS X-Ray.

Metrics

By default, all Artillery metrics will be sent to CloudWatch. Each Artillery metric will create a custom CloudWatch metric (opens in a new tab), which incur extra AWS fee for each metric. You can configure a specific list of metrics to send with the includeOnly setting (see Configuration section below).

You can learn more about AWS CloudWatch custom metric pricing in this blog post from Vantage.sh: https://www.vantage.sh/blog/cloudwatch-metrics-pricing-explained-in-plain-english (opens in a new tab)

Configuration options

Example configuration

The following configuration enables CloudWatch reporting. All metrics generated by the test will be sent to us-east-1 region, and tagged with extra dimensions to specify the team running the tests (SQA), and the name of the service under test (checkout-svc).

config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: us-east-1
        namespace: continous-testing
        dimensions:
          - name: Team
            value: SQA
          - name: Service
            value: checkout-svc
NameDescription
regionCloudWatch region where metrics will be sent.
namespaceMetric namespace. Defaults to "artillery".
nameThe value of the default Name dimension attached to every metric. Defaults to "loadtest".
excludedA list of metric names which should not be sent to CloudWatch. Defaults to an empty list, i.e. all metrics are sent to CloudWatch.
includeOnlyA list of specific metrics to send to CloudWatch. No other metrics will be sent. Defaults to an empty list, i.e. all metrics are sent to CloudWatch.
dimensionsA list of extra dimensions to attache to every metric as name/value pairs. Defaults to an empty list.
traces
Added inv2.0.9
Set to send traces to AWS X-Ray. See traces specific configuration

Traces

Added inv2.0.9

Powered by: OpenTelemetry reporter

⚠️

Experimental This feature is under active development. We aim to keep it stable, but it may need to introduce breaking changes. Please open an issue (opens in a new tab) or discussion (opens in a new tab) for any feedback.

With the CloudWatch reporter you can also send traces to AWS X-Ray (opens in a new tab). Tracing is available for both HTTP and Playwright engines, and the tracing data available is different depending on the engine used.

An additional plugins.publish-metrics.spans.exported metric will be recorded and will appear in your reports when tracing is enabled. It represents the number of spans exported to AWS X-Ray.

Configuration options

CloudWatch requires a running instance of the CloudWatch Agent (opens in a new tab) (or the AWS Distro for OpenTelemetry (opens in a new tab)) in order to ingest traces. When you run a test on AWS Fargate, Artillery will take care of this for you automatically, but for testing on your local machine you will need to run the agent yourself.
Note: AWS Lambda is not currently supported.

Example configuration
plugins:
  publish-metrics:
    - type: 'cloudwatch'
      traces:
        sampleRate: 0.5
        replaceSpanNameRegex:
          - pattern: get_user_[a-zA-Z0-9]+
            as: get_user_id
        annotations:
          testType: LoadTest
          tool: Artillery
NameDescription
serviceNameName of your service. Defaults to Artillery-test
sampleRateSample rate. Percentage of traces to send represented by a value between 0 and 1. (defaults to 1 - all traces are sent)
useRequestNamesIf set to true the request names provided in test script will be used as span names
annotationsCustom annotations to be added to each span in key: value pairs.
replaceSpanNameRegexA list of replacement maps that consist of:
pattern - regex pattern to match against the span names
as - string to replace the matched pattern in the span names

AWS Fargate - additional setup

Artillery sets up the AWS Distro for OpenTelemetry (opens in a new tab) configured with the AWS X-Ray exporter as a sidecar container.

Important: If you ran Artillery tests on AWS Fargate in the past on the account you are using, you will need to delete the existing artillery-worker-role IAM role and artilleryio-ecs-worker-policy IAM policy from your account prior to using this feature.


Artillery will automatically recreate the role and policy with the correct permissions to send traces to AWS X-Ray.

To run the test run:

artillery run-fargate scenario.yaml

Local runs - additional setup

You'll need to install and configure a CloudWatch Agent (opens in a new tab) or AWS Distro for OpenTelemetry (opens in a new tab) to ingest OTLP traces:

CloudWatch Agent

"traces": {
  "traces_collected": {
    "otlp": {}
  }
}

AWS Distro for OpenTelemetry

Debugging

Set DEBUG=plugin:publish-metrics:cloudwatch when running your tests to print out helpful debugging messages when sending metrics to Datadog.

DEBUG=plugin:publish-metrics:cloudwatch artillery run my-script.yaml

Examples

config:
  plugins:
    publish-metrics:
      - type: cloudwatch
        region: us-east-1
        namespace: continous-testing
        dimensions:
          - name: Team
            value: SQA
        traces:
          serviceName: my-service
          useRequestNames: true
          annotations:
            testType: LoadTest
            tool: Artillery