From day one, our vision with Artillery & Artillery Pro has been making performance testing as easy and frictionless as possible. We want to remove as many barriers to running load tests, at any scale, and with the frequency that a development team requires (ideally as part of a CI/CD pipeline). Modern production systems keep growing in complexity, and to increase confidence that a code or configuration change isn’t going to severely impact performance in production, those changes need to be load tested automatically before they reach production.
Today we are excited to announce another big step towards realizing that vision: first class AWS Fargate support in Artillery Pro. AWS Fargate provides a way to run containers on demand, with VMs provisioned and managed transparently and securely by the underlying AWS platform. What it means for you as an Artillery Pro user is that you can go from running a test from your local machine, to running it from the cloud at 10x, 50x or 100x the scale in a matter of minutes, on demand, without needing to set up or manage any infrastruture, and paying only for the AWS resources your tests require while they are running.
How do we use Fargate with Artillery Pro?
If your team is already using Artillery Pro, you need to upgrade to v1.3.0 and then tweak your invocation of
artillery run-cluster as follows:
- Add a
--fargateflag to use Fargate
- Specify the subnets which should be used for worker containers with
- There is no step 3!
You don’t need to change anything else in your script or in how you run it. Everything will work in the exact same way.
Please see the documentation for more details.
Let’s look at the rough cost of running a test with Artillery Pro + Fargate.
Fargate is billed per-second (with a minimum charge of one minute). Let’s say you run your tests from
us-east-1, and you want to run a 45 minute long test at large load (sometimes referred to as soak testing), using 20 workers to generate load. The cost to run that test on Fargate using the default configuration of containers with 1 vCPU and 2GB of memory would then be:
($0.0506 per vCPU/hour + ($0.0127 per GB/memory * 2)) * (45/60) = (0.0506 + (0.0127 * 2)) * 0.75 = $0.057
(Source: Fargate Pricing)
That’s less than 6 cents to run a 45-minute load test with 20 workers. If you run twenty of those tests every business day, you’d pay just $22.80/month in Fargate costs. And low cost is not the only great thing about this, it’s also worth noting that unlike other load testing solutions, Artillery Pro does not place artificial limits on your usage such as limiting the max number of virtual users in a test, max number of connections, the number of test-hours you’re allowed to use, or the max number of requests your tests may make.
We believe that development teams should be able to run performance tests as part of their CI/CD pipeline, frequently, and at the scale they need, without having to worry about astronomic costs or exceeding some artificial limit imposed by the load testing platform they are using. Thanks to Fargate, any team with an AWS account can now run load tests, at the scale they need, and without setting up or managing any infrastructure.
Classic ECS Is Still Supported
Most of our customers already run large classic ECS clusters (i.e. clusters composed of EC2 container instances) with plenty of spare capacity to run load tests. Those teams may not need to or want to use Fargate, and these changes do not affect how Artillery Pro runs tests on existing ECS clusters in any way. The new functionality is there to give all of our users even more flexibility, and we will continue to support existing functionality as normal.
Try Artillery Pro with Fargate
If you’re an existing Artillery Pro user, upgrade to v1.3.0, and use the new Fargate-specific flags for
run-cluster command. As always, if you have any questions, don’t hesitate to drop us a line on firstname.lastname@example.org.
If you’d like to try out Artillery Pro for your team, drop us a line to get a free Artillery Pro trial.