Socket.IO Engine

Socket.IO (opens in a new tab) is a popular library for building real-time event-based applications. This page covers Socket.IO testing functionality in Artillery provided by the built-in Socket.IO engine.

Enabling Socket.IO support

To use the Socket.IO engine in an Artillery scenario, set the engine attribute of a scenario definition to socketio.

scenarios:
  - name: My Socket.IO test
    engine: socketio # Enable the Socket.IO engine
    flow:
      - emit:
          channel: 'echo'
          data: 'Hello world!'

Socket.IO-specific configuration

Query

Query parameters (opens in a new tab) can be specified as a string or as a dictionary:

config:
  target: 'https://myapp.staging:3002'
  socketio:
    query: 'user_id=0xc0ffee&color=blue'

config:
  target: 'https://myapp.staging:3002'
  socketio:
    query:
      user_id: '0xc0fee'
      color: 'blue'

Path

A custom path (opens in a new tab) may be set with the path option.

In the following example, the virtual users will connect to the admin namespace with the custom path mypath:

config:
  target: 'https://myapp.staging:3002/admin'
  socketio:
    path: '/mypath'

Extra headers

Extra headers (opens in a new tab) may be passed with extraHeaders option:

config:
  target: 'https://myapp.staging:3002/admin'
  socketio:
    extraHeaders:
      x-client-id: 'abc'

The extraHeaders option only works if the default polling transport is used. When using other transports, extra headers won't be taken into account by the server.

WebSocket transports only

You can skip long-polling (opens in a new tab) by using the transports option to specify WebSocket transport:

config:
  target: 'https://myapp.staging:3002/admin'
  socketio:
    transports: ['websocket']

Scenario actions and configuration

The Socket.IO engine allows for HTTP actions actions to be used in scenarios alongside emit, which is the main Socket.IO action. Looping with loop and pausing execution with think is also supported, see Testing HTTP for details.

`emit`

Sending Data

The Socket.IO engine now follows the SocketIO convention: the emit action is an array, where the first argument is the channel, and any subsequent arguments are your data. Data can be a string or object (or more generally, any serializable data structure).

Alternatively, the emit action still has backwards-compatibility and supports the following attributes:

channel - The name of the Socket.IO channel to emit an event.
data - The data to emit as a string.

Namespacing

If you want to use an optional namespace for emitting the event, you must specify it alongside emit. See example below.

Response Validation

To validate a response, set a response object at the same level of emit. It takes the following attributes:

channel - The name of the channel where the response is received.
data - The data to verify is in the response.
match - Match the response with a JSONPath expression. This attribute expects an object with two attributes:
- json - The part of the response to compare.
- value - The expected value.
on - Can be used instead of channel to listen to a specific event. Added inv2.0.1
args - Can be used instead of data to assert that the response emits these arguments. Will typically be an array of strings, but can also be an object or string. Added inv2.0.1

If you emit to a specific namespace, the response data is expected within the same namespace.

Acknowledgements

To validate a Socket.io acknowledgement response (opens in a new tab), set acknowledge at the same level as emit. It takes the following attributes:

data - Match the response exactly to the value provided.
match - Match the response with a JSONPath expression. This attribute expects an object with two attributes:
- json - The part of the response to compare.
- value - The expected value.
args - Can be used to validate the acknowledge callback arguments, and will take the same shape as them. Added inv2.0.1

Examples

Emit a single event

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit an event'
    engine: socketio
    flow:
      - emit:
          - 'echo' # your channel
          - 'Hello from Artillery'

Or, if you want, you can still use the old interface:

- emit:
    channel: 'echo'
    data: 'Hello from Artillery'

Emit multiple arguments

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit an event'
    engine: socketio
    flow:
      - emit:
          - 'echo' # your channel
          - 'Hello'
          - 'from'
          - 'artillery'

Emit an event and validate a response

Validate the response using data

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit an event'
    engine: socketio
    flow:
      - emit:
          channel: 'echo'
          data: 'Hello from Artillery'
        response:
          channel: 'echoResponse'
          data: 'Hello from Artillery'

Validate the response on a specific event using args

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit an event'
    engine: socketio
    flow:
      - emit:
          channel: 'echo'
          data: 'Hello from Artillery'
        response:
          on: 'echoResponse'
          args:
            - 'Hello from Artillery'

Emit an event and validate an acknowledgement

Validate the acknowledgement using data

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit and validate acknowledgement'
    engine: socketio
    flow:
      - emit:
          channel: 'userDetails'
        acknowledge:
          match:
            json: '$.0.name'
            value: 'Artillery'

Validate an acknowledgement using args

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit and validate acknowledgement'
    engine: socketio
    flow:
      - emit:
          channel: 'userDetails'
        acknowledge:
          args:
            name: 'Artillery'

Emit data to a namespace

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 60
      arrivalRate: 25
 
scenarios:
  - name: 'Emit to /namespace1'
    engine: socketio
    flow:
      - namespace: '/namespace1'
        emit:
          channel: 'echo'
          data: 'Emitting data to namespace'

Mixing in HTTP Actions

A single scenario can combine both HTTP and Socket.IO actions.

The following example is a common scenario when testing servers based on Express.js (opens in a new tab):

config:
  target: 'http://myapp.example.com'
  phases:
    - duration: 10
      arrivalRate: 1
 
scenarios:
  - engine: 'socketio'
    flow:
      - get:
          url: '/movies/10'
          capture:
            - json: '$.title'
              as: 'title'
            - json: '$.genre'
              as: 'genre'
      - log: 'Emitting captured values: {{ title }}, {{ genre }}'
      - emit:
          channel: 'echo'
          data: '{{ title }}'
          # validate the received response
        response:
          channel: 'echoResponse'
          data: '{{ title }}'
      - emit:
          channel: 'echo'
          data: '{{ genre }}'
        response:
          channel: 'echoResponse'
          data: '{{ genre }}'
      - think: 1
      - emit:
          channel: 'echo'
          data: 'do not care about the response'
      - emit:
          channel: 'echo'
          data: 'still do not care about the response'
      - think: 1

Connecting Without Sending Any Data

You can use think to emulate clients connecting to the app and listening without sending anything themselves.

config:
  target: 'https://socketio.test/'
  phases:
    - duration: 3600
      arrivalRate: 5
scenarios:
  - engine: 'socketio'
    flow:
      - think: 600 # do nothing for 10m and disconnect

Metrics reported by the engine

In addition to the default metrics reported by Artillery, the Socket.io engine reports the following metrics:

Metric	Type	Description
`socketio.emit`	Counter (count)	Number of emitted messages.
`socketio.response_time.<aggregation>`	Histogram (milliseconds)	Time taken from emitting a message to receiving a response.
`socketio.emit_rate`	Rate (msg/sec)	Rate of Socket.io messages emitted over the time period.

Additionally, since it's possible to use the HTTP engine together with Socket.io, you will also see the HTTP metrics reported.

Debugging

If you're having issues getting your test scenarios to complete successfully, you can print out helpful debugging messages using the DEBUG environment variable.

Print errors

Set DEBUG=socketio when running your tests to view any errors that occurred during the test run.

On macOS and Linux systems, you can temporarily set the DEBUG environment variable by setting its value when running your Artillery test script:

DEBUG=socketio artillery run my-script.yaml

For the Windows Command Prompt, you first need to set the DEBUG environment variable using the set command before running the test script:

set DEBUG=socketio
artillery run my-script.yaml

If you use PowerShell on Windows, you can set the DEBUG environment variable using $Env:DEBUG before running the test script:

$Env:DEBUG = 'socketio'
artillery run my-script.yaml

Playwright WebSocket