How To Improve Testing Framework Performance
Testing framework is used to allow maintainability of your tests. In E2E testing we usually use some of the well known patterns for automated execution of the tests. When we talk about performance of the tests we are concerned about infrastructure, flakiness, application loading times, etc. There are many things which influence the test performance. The optimization efforts usually target improvements of the infrastructure, improving the application under test, parallelization, etc. However, most people don’t think how the design of your framework affects the performance. Here are some of the ways to improve testing framework performance.
Data setup
The first thing we will discuss is test data setup. It is not uncommon to see that there are hard coded values for data in the tests. Another option is to set data through UI using the test script in some before block. The best way to setup data from performance perspective is to send data to the database through API and then work with that data in UI. The most common testing libraries and tools offer this functionality. Let’s see how this works in Cypress.
For demonstration purposes I have here one simple test which at first adds one item in the cart. Then the test checks if the item is added. I do it all through UI.
Now, a better approach would be to use the API to send the item to the cart and then check the cart.
Now let see what did we achieve with this. The first test finished in 4 seconds, while the second one finished in 3 seconds.
One second seems like a small improvement, but it was a fast and short test in the first place. For more complex tests the benefits are even higher.
User journey optimization
Another common issue in writing tests is clicking through UI as user would do to get to a certain page. In the above example I open first the home page and then after I add the item to the cart I click on cart button. TO improve our testing framework performance a better approach would be to go directly to cart page after sending the request with data. Let’s see that in action.
And now after this change lets run another test and see what are the results in execution times. Remember the first test completed in 4 seconds as shown above.
The test completed in 1 second. If I compare that to the initial test, we’ve made 75% improvement with two simple changes in the test design.
Conclusion
Simple design changes can literally improve the testing framework performance by a lot. We haven’t created any parallelization, we haven’t done any infrastructure improvements, no optimization of the system under test. The only thing I did was a simple change of my thinking about the test. I showed before already how to improve the performance of the tests with reusable sessions which is also a great technique. The advice from these two articles are very helpful for optimizing the performance of your testing framework.
