Skip to content

Latest commit

 

History

History
 
 

bench

README.md

DCLedger Load Testing

DCLedger testing is implemented in python3 and based on Locust framework.

Requirements

Installation

Run (consider to use virtual environment):

pip3 install -r bench/requirements.txt

Optional If you need to monitor server-side metrics please install Prometheus.

Preparation

  • If you run load tests on a local pool, follow the steps below (this item is needed only for a local pool):

    • Each write transactions is signed and thus requires:

      • an account with write permissions (e.g. Vendor account)
      • proper values for txn sequence which enforces txns ordering for an account

    • Initialize the pool and test accounts (Warning applicable to local in-docker pool only for now):

make localnet_clean
make localnet_init
make localnet_start
# Note: once started ledger may require some time to complete the initialization.

  • Copy accounts keys to folder ~/.dcl/keyring-test:

    • After connecting to the network, we must copy the Trustee account keys to ~/.dcl/keyring-test. The number of accounts with role Trustee must not exceed 3.

    • If you run the test against a local pool, the account keys can be copied as follows:

      cp ./.localnet/node0/keyring-test/* ~/.dcl/keyring-test

  • Go to bench folder

    cd bench

  • Open the .env file:

    • update fields. e.g.:

      DCLD_VERSION=v0.11.0
DCLD_NODE=tcp://host.docker.internal:26657
DCLD_CHAIN_ID=dclchain

WRITE_HOSTS=http://host.docker.internal:26657
READ_HOSTS=http://host.docker.internal:26640
TRUSTEE_ACCOUNT_NAME=jack
COUNT_USERS=4

CSV_REPORT_FILE_NAME=example
HTML_REPORT_FILE_NAME=example

      <DCLD_VERSION> - dcld binary version. <DCLD_NODE> - Address <host>:<port> of the node to connect. This node needs for adding account with role Vendor in write load tests. Default value in local in docker nodes is tcp://host.docker.internal:26657 <DCLD_CHAIN_ID> - unique chain ID of the network you are going to connect. <WRITE_HOSTS> - hosts for writing load tests. Default value in local in docker nodes is http://host.docker.internal:26657 <READ_HOSTS> - hosts for reading load tests. Default value in local in docker nodes is http://host.docker.internal:26640 <TRUSTEE_ACCOUNT_NAME> - trustee account name. Trustee account, which will add account with Vendor role in write load tests. <COUNT_USERS> - number of users in load tests. Number of users should be equal to number of workers for write tests. <CSV_REPORT_FILE_NAME> - generates a set of stat files (summary, failures, exceptions and stats history). <HTML_REPORT_FILE_NAME> - generates an html report Web UI also includes Download Data tab where the reports can be found.

      Note: The above options can either point to local in docker nodes or to external nodes.

  • Build docker-compose file

    docker compose build

Run

(Optional) Launch Prometheus

prometheus --config.file=bench/prometheus.yml

And open http://localhost:9090/ to query and monitor the server-side metrics.

Headless

To run write load tests

  • Open the docker-compose.yml and add a new command for master and worker.
    e.g.: for master

    command:
  - "--headless"
  - "WriteModelLoadTest"

    e.g.: for worker:

    command:
  - "--headless"
  - "WriteModelLoadTest"

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine. Number of users should be equal to number of workers for write tests.

To run read load tests

  • Open the docker-compose.yml and add a new command for master and worker.
    e.g.: for master

    command:
  - "--headless"
  - "ReadModelLoadTest"

    e.g.: for worker:

    command:
  - "--headless"
  - "ReadModelLoadTest"

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine.

To run write/read load tests

  • Open the docker-compose.yml and add a new command for master and worker.
    e.g.: for master

    command:
  - "--headless"

    e.g.: for worker:

    command:
  - "--headless"

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine. Number of users should be equal to number of workers for write tests.

Web UI

To run write load tests

  • Open the docker-compose.yml and add a new command for master and worker. e.g.: for master

    command:
  - "WriteModelLoadTest"

    e.g.: for worker:

    command:
  - "WriteModelLoadTest"

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine. Number of users should be equal to number of workers for write tests.

To run read load tests

  • Open the docker-compose.yml and add a new command for master and worker.
    e.g.: for master

    command:
  - "ReadModelLoadTest"

    e.g.: for worker:

    command:
  - "ReadModelLoadTest"

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine.

To run write/read load tests

  • Open the docker-compose.yml and add a new field extra_hosts for master and worker (this item need only for the local pool): e.g.: for master

    extra_hosts:
  - "host.docker.internal:host-gateway"

    e.g.: for worker

    extra_hosts:
  - "host.docker.internal:host-gateway"

    <host.docker.internal:host-gateway> - need to connecting to localhost from locust container.

  • Run docker-compose.yml

    docker compose up --scale worker=<workers-count>
# The workers run your Users and send back statistics to the master. The master instance doesn't run any Users itself. Both the master and worker machines must have a copy of the locustfile when running Locust distributed.

    <workers-count>- number of machine. Number of users should be equal to number of workers for write tests.

Then you can open http://localhost:8089/ and launch the tests from the browser.

More details can be found in:

Re-run

Warning applicable to local in-docker pool only for now

Next time when you run the test using the same data you will likely get many (all) failures since DCLedger will complain about already written data or wrong sequence numbers.

For that case you may consider to reset the ledger as follows:

make localnet_reset localnet_start

FAQ

locust may complain about ulimit values for open files

Please check the details here.

Additional sources (linux):

ToDo

  • consider different types of tx: async, sync (currently used), commit