Edit

Share via

Key concepts for new Azure Load Testing users

  • Article

Learn about the key concepts and components of Azure Load Testing. This information can help you to more effectively set up a load test to identify performance issues in your application.

General concepts of load testing

Learn about the key concepts related to running load tests.

Virtual users

A virtual user runs a particular test case against your server application and runs independently of other virtual users. You can use multiple virtual users to simulate concurrent connections to your server application.

Apache JMeter also refers to virtual users as threads. In the JMeter test script, a thread group element lets you specify the pool of virtual users. Learn about thread groups in the Apache JMeter documentation.

The total number of virtual users for your load test depends on the number of virtual users in the test script and the number of test engine instances.

The formula is: Total virtual users = (virtual users in the JMX file) * (number of test engine instances).

You can achieve the target number of virtual users by configuring the number of test engine instances, the number of virtual users in the test script, or a combination of both.

Ramp-up time

The ramp-up time is the amount of time to get to the full number of virtual users for the load test. If the number of virtual users is 20, and the ramp-up time is 120 seconds, then it takes 120 seconds to get to all 20 virtual users. Each virtual user will start 6 (120/20) seconds after the previous user was started.

Response time

The response time of an individual request, or elapsed time in JMeter, is the total time from just before sending the request to just after the last response is received. The response time doesn't include the time to render the response. Any client code, such as JavaScript, isn't processed during the load test.

Latency

The latency of an individual request is the total time from just before sending the request to just after the first response is received. Latency includes all the processing needed to assemble the request and assembling the first part of the response.

Requests per second (RPS)

Requests per second (RPS), or throughput, is the total number of requests to the server application that your load test generates per second.

The formula is: RPS = (number of requests) / (total time in seconds).

The time is calculated from the start of the first sample to the end of the last sample. This time includes any intervals between samples, for example if the test script contains timers.

Another way to calculate the RPS is based on the average application's latency and the number of virtual users. To simulate a specific number of RPS with a load test, given the application's latency, you can then calculate the required number of virtual users.

The formula is: Virtual users = (RPS) * (latency in seconds).

For example, given an application latency of 20 milliseconds (0.02 seconds), to simulate 100,000 RPS, you should configure the load test with 2,000 virtual users (100,000 * 0.02).

Azure Load Testing components

Learn about the key concepts and components of Azure Load Testing. The following diagram gives an overview of how the different concepts relate to one another.

Diagram that shows how the different concepts in Azure Load Testing relate to one another.

Load testing resource

The Azure load testing resource is the top-level resource for your load-testing activities. This resource provides a centralized place to view and manage load tests, test results, and related artifacts.

When you create a load test resource, you specify its location, which determines the location of the test engines. Azure Load Testing automatically encrypts all artifacts in your resource. You can choose between Microsoft-managed keys or use your own customer-managed keys for encryption.

To run a load test for your application, you add a test to your load testing resource. A resource can contain zero or more tests.

You can use Azure role-based access control to grant access to your load testing resource and related artifacts.

Azure Load Testing lets you use managed identities to access Azure Key Vault for storing load test secret parameters or certificates. You can use either a user-assigned or system-assigned managed identity.

Test

A test describes the load test configuration for your application. You add a test to an existing Azure load testing resource.

A test contains a test plan, which describes the steps to invoke the application endpoint. You can define the test plan in one of three ways:

Azure Load Testing supports all communication protocols that JMeter supports, not only HTTP-based endpoints. For example, you might want to read from or write to a database or message queue in the test script.

Azure Load Testing currently does not support other testing frameworks than Apache JMeter and Locust.

The test also specifies the configuration settings for running the load test:

In addition, you can upload CSV input data files and test configuration files to the load test.

When you start a test, Azure Load Testing deploys the test script, related files, and configuration to the test engine instances. The test engine instances then initiate the test script to simulate the application load.

Each time you start a test, Azure Load Testing creates a test run and attaches it to the test.

Test run

A test run represents one execution of a load test. When you run a test, the test run contains a copy of the configuration settings from the associated test.

After the test run completes, you can view and analyze the load test results in the Azure Load Testing dashboard in the Azure portal.

Alternately, you can download the test logs and export the test results file.

Important

When you update a test, the existing test runs don't automatically inherit the new settings from the test. The new settings are only used by new test runs when you run the test. If you rerun an existing test run, the original settings of the test run are used.

Test engine

A test engine is computing infrastructure, managed by Microsoft that runs the test script. The test engine instances run the test script in parallel. You can scale out your load test by configuring the number of test engine instances. Learn how to configure the number of virtual users, or simulate a target number of requests per second.

The test engines are hosted in the same location as your Azure Load Testing resource. You can configure the Azure region when you create the Azure load testing resource.

While the test script runs, Azure Load Testing collects and aggregates the testing framework logs from all test engine instances. You can download the logs for analyzing errors during the load test.

App component

When you run a load test for an Azure-hosted application, you can monitor resource metrics for the different Azure application components (server-side metrics). While the load test runs, and after completion of the test, you can monitor and analyze the resource metrics in the Azure Load Testing dashboard.

When you create or update a load test, you can configure the list of app components that Azure Load Testing will monitor. You can modify the list of default resource metrics for each app component.

Learn more about which Azure resource types that Azure Load Testing supports.

Metrics

During a load test, Azure Load Testing collects metrics about the test execution. There are two types of metrics:

  • Client-side metrics are reported by the test engines. These metrics include the number of virtual users, the request response time, the number of failed requests, or the number of requests per second. You can define test fail criteria based on these client-side metrics.

  • Server-side metrics are available for Azure-hosted applications and provide information about your Azure application components. Azure Load Testing integrates with Azure Monitor, including Application Insights and Container insights, to capture details from the Azure services. Depending on the type of service, different metrics are available. For example, metrics can be for the number of database reads, the type of HTTP responses, or container resource consumption.

Related content

You now know the key concepts of Azure Load Testing to start creating a load test.