Extend Your Performance Tests With Open Model Thread Group

Today we will focus on performance testing again but this time we use Jmeter. Specifically let’s talk about the ability to create a Jmeter test script that can be repeated with different loads. The version 5.5 of Jmeter is not new, but it came with a new version of Thread Group. It supports Open Model Thread Group which uses arrivals for setup just like Arrivals Thread Group but with one difference. With Open Model Thread Group we can dictate the rate of arrivals. You can even change the number of arrivals for different iterations which was not possible with the old thread groups.

The terminology

As already mentioned Open Model Thread Group works with “arrivals”. What is an arrival? Jmeter documentation describes it as “thread iteration start” but in simple terms we can say that it is just requests arriving in the system. Then Arrival rate must be a rate at which these requests arrive.

Setup of Open Model Thread Group

This type of thread group comes with installation of Jmeter 5.5 or later. There are several options you can set and setup itself is quite easy because you just need to click on preset options. There are also control options to choose what do you want to happen when sampler error occurs. These options are not new nor related specifically to this thread group type.

The options to control the arrivals are below. We refer to these options as Schedule according to the section in which they reside. Rate controls the speed of arrivals. The preset unit of measurement is set to minutes for all controls but it can be changed to seconds or hours. Random arrivals is a breakpoint where some change in the script would happen and it describes the timeout after which this change should happen. For example, we started the test and after 5 minutes we want to change the rate to 20 arrivals. To change it we would use random_arrivals(5 min). Let’s see a practical example of a simple setup. The current setup is like this

rate(5/min) 
random_arrivals(5 min) rate(20/min) 
random_arrivals(15 min) 
random_arrivals(1 min) rate(0/min)

To explain the above code snippet. We start the test with rate of 5 arrivals on each minute (rate(5/min) command). After 5 minutes (random_arrivals(5 min) command) we change the rate to 20 arrivals per minute(rate(20/min) command). For the next 15 minutes we hold the rate (random_arrivals(15 min) command) and then we wait for one minute before we drop the rate to 0.

More complex setup

Throughout my experience I have seen teams performing a test run then waiting for a few minutes for the server to recover and repeating the tests for few more times and then combining the results manually. Some teams make warm-up test runs to prepare the server for the main load. I’ve seen also teams doing test runs by gradually increasing the load. All of these examples have one thing in common – a lot of manual re-running the tests. Some of these scenarios, depending on the situation, require making changes in the test script before test run. With Open Model Thread Group you can have all these scenarios in a single test run.

Let’s repeat the same test as above but with three different maximum number of users: 20 users at the beginning, then pause and repeat the test with 50 users and then pause again and finally repeat the test with 100 users.

In this way you can even combine a different types of performance testing. For example you can simulate steady load and then add sudden spikes of load.

Usage of scripting

You can use Jmeter variables in the setup and in this way you can use environment variables from your pipeline, if you have performance tests setup in the CI/CD. In the below example I have set variables initialRate, holdRate and timeout and for each variable I have added default value 5, 20 and 5 again respectively.

rate(${__P(initialRate,5)}/min) 
random_arrivals(5 min) rate(${__P(holdRate,20)}/min) 
random_arrivals(15 min) 
random_arrivals(1 min) rate(0/min)
pause(${__P(timeout,5)} min)

Usage of Groovy is also possible as in the following example. For each of the iterator values (1,2,3) multiply the iterator with 10 to get the starting rate. Then multiply the iterator with 100 to get the time until the first change. Next, multiply the iterator with 10 to get the value of pause between test iterations.

Conclusion

The benefits of this type of thread group are enormous. It combines several thread groups we had before and offers a lot of flexibility. Finally we don’t need to adjust the tests manually before re-running them which is especially cumbersome in situation when the tests execute in CI/CD. Now we can use a specific configuration which covers all our needs. We can use preset values during the setup or we can edit them manually. For advanced usage, Jmeter gave us an option to use scripting in the scheduler and extended the flexibility of these thread group even further.