Artillery Pro Changelog


v2.2.2

Date: November 17, 2020

Fixes & Improvements

  • Fix an issue which could prevent more than 3 concurrent test runs under some conditions

v2.2.1

Date: October 27, 2020

Fixes & Improvements

  • Address third-party dependency deprecation notices and other issues
  • Minor performance improvements
  • Fix issues when running tests with >100 workers under some conditions

v2.2.0

Date: March 31, 2020

Please make sure update the Artillery Pro backend after upgrading the CLI before running any tests.

New Features

  • Artillery Pro tests may now be run from private subnets with no public internet access with AWS PrivateLink to enable running Artillery Pro tests in highly sensitive “air gapped” environments.
  • [run-test] The --region option is now optional. The region that the Artillery Pro backend is deployed to will be used when running tests.
  • [run-test] The --public-subnet-ids option is now optional when running tests on Fargate. If not set, public subnets in the default VPC will be used.
  • [run-test] The --public-subnet-ids option has been replaced by --subnet-ids and is scheduled for deprecation in a future release.
  • [run-test] ulimit settings passed to containers may now be customized via --launch-config option. The default nofile limit is now set to 8192 to support high-concurrency tests.

Fixes & Improvements

  • Startup time improvements for tests running in EU and AP regions

v2.2.0-3

Date: December 19, 2019

This is an early-access release of Artillery Pro for users who would like early access to most recent features. If you run into any issues, please get in touch on support@artillery.io.

New Features

Fixes & Improvements

  • Fixed an issue with could sometimes stop tests that make use of a large number of npm dependencies from starting up correctly.

v2.2.0-2

Date: December 13, 2019

Maintenance release updating third-party dependencies to address deprecation notices and CVE advisories.


v2.2.0-1

Date: October 14, 2019

This is an early-access release of Artillery Pro for users who would like early access to most recent features. If you run into any issues, please get in touch on support@artillery.io.

Fixes & Improvements

  • Fixed an issue with setting NPM_REGISTRY to a custom URL
  • Fixed RPS values reported by run-test command being lower than actual values under some circumstances

v2.2.0-0

Date: October 9, 2019

This is an early-access release of Artillery Pro for users who would like early access to most recent features. If you run into any issues, please get in touch on support@artillery.io.

New Features

  • Private NPM registries are now supported - configure access to private npm modules
  • run-cluster command has been renamed to run-test. Either may still be used to run tests, but run-cluster will be deprecated at some point in the future in favor of run-test.
  • Add set-config-value command which allows for various Artillery Pro configuration values to be set

Fixes & Improvements

  • CSV payload paths may now contain variables
  • Various performance improvements in the CLI and workers
  • An issue with create-test which could result in incorrect test names to be set under some conditions

v2.1.3

Maintenance release updating third-party dependencies to address deprecation notices and CVE advisories.


v2.1.2

Fixes

  • Various fixes for auto-detection of npm modules and other assets required by test scripts.

v2.1.1

Fixes

  • Fix for artillery deploy where --license-key-file option would not work as expected under some conditions

v2.1.0

New Features

  • Yarn is now supported. If a yarn.lock file can be found, yarn will be used to install test dependencies by the workers instead of npm
  • run-cluster and create-test commands will now list the files which will get packaged up and made available to workers

Fixes

  • Fix a bug with handling of @scoped dependencies in custom JS code

v2.0.0

New Features

  • 9 more AWS regions are now supported. Artillery Pro can now be deployed to any of the following 13 regions: us-east-1, us-east-2, us-west-2, eu-west-1, us-west-1, ap-south-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-northeast-1, eu-central-1, eu-west-2, or ca-central-1.
  • activate-pro command has been deprecated to make Artillery Pro CLI feel more like the official AWS CLI. End-users of the CLI such as developers or testers using run-cluster as well as machine users such as CI/CD jobs will no longer need to activate their installation of Artillery Pro CLI, as long as they have adequate IAM permissions to interact with the Artillery Pro backend deployed to their team’s AWS account.
  • setup-pro command has been renamed to deploy. Any existing scripts using setup-pro will continue working, but setup-pro but will be deprecated in a future release.
  • config.includeFiles can be used to provide a list of files to package up along the test scripts and any custom JS functions
  • New subscription command to check subscription status and to request license keys

Optimisations And Fixes

  • Better progress and error messages in run-cluster
  • Worker Docker image size reduced by 25% - faster startup time on Fargate
  • Default memory allocation reduced on ECS
  • Workers now run Node.js 12.2.0 which offers better performance
  • Improved error handling and signaling in workers
  • Improved CloudWatch logging from workers
  • Fix for --overrides in run-cluster which could lead to workers crashing under certain conditions
  • Fix for plugins not always being detected correctly
  • Fix for deploy failing under certain condition with IAM
  • Support HTTP responses where headers exceed 8kb in size
  • Fix for a race condition in run-cluster which could lead to CLI not detecting that a test run has completed successfully
  • Better detection of ECS/Fargate scheduling errors
  • Various other minor performance improvements in the CLI and the backend

v1.3.3

  • Artillery workers will now use 1024 CPU units (equivalent to 1 vCPU) and 2048 memory units by default on both Fargate and classic ECS to improve performance
  • Fixes an issue where the CLI would sometimes hang indefinitely due to getting into an inconsistent internal state because of Fargate taking too long to provision required containers
  • The CLI will now provide progress updates as Fargate provisions the required resources (number of tasks ready / being provisioned / pending)
  • Improvements to the accuracy of intermediate reports
  • Improves Node v11 support

v1.3.2

  • Fix for cpu/memory overrides with --launch-config sometimes not being set correctly when running on Fargate
  • Fix for subscription limits sometimes not being applied correctly
  • Activation with a Github account now requires the use of the --github flag
  • Artillery Pro will now print a notification when an update is available
  • Upgrade worker containers to use Node.js 10
  • Various improvements to error handling & providing friendlier error messages

v1.3.1

  • Stability improvements for the installation/upgrade process of backend components
  • Speed improvements in metric aggregation & reporting

v1.3.0

  • Serverless load testing with Fargate is here! To use Fargate, add a --fargate flag, and tell Fargate which public subnets to use with --public-subnet-ids, and your test will run from serverless Fargate containers rather than managed EC2 instances. Any existing Artillery test script, any scale, without having to manage any EC2 VMs.
  • Fixed a bug which could cause run-cluster to hang and never exit despite the test completing successfully.
  • Various internal stability and performance improvements.

Important - Artillery Pro backend components need to be upgraded with artillery setup-pro before using v1.3.0.


v1.2.7

  • ECS worker containers can be customized now (with ECS container overrides for cpu, memory, and environment) with –launch-config
  • Custom DEBUG variable may be set to customize debug output in CloudWatch logs for a test run
  • Alpha AWS Fargate support (get in touch on team@artillery.io if you’d like to give it a go)
  • Various internal improvements

v1.2.6

  • Autodetect plugins used in a test script and make them available in worker containers running on ECS
  • Performance improvements

v1.2.5

  • File uploads: setContentLengthHeader option may now be set
  • Non-flat custom JS code dependencies are resolved properly now (for test scripts that load a custom JS module which loads other local modules in turn)