BlazeMeter Multi-Test shows a mismatch between user count provided in test and report

Products

• BlazeMeter

Issue/Introduction

When running a multi-test (10  test scenarios, totaling 20 users, 2 user per script), the result file Max Users is only 6. However, if we look at the Max User count for each individual test scenario, we see all 20 users, when summing all the Max User counts individually.

Environment

Release : SAAS

Component : BLAZEMETER PERFORMANCE TESTING

Resolution

R&D did a thorough investigation into the behavior of multi-test, and found that this is not a defect. Below is their explanation of the findings:

"The meaning of maximum concurrency of all scenarios is the maximum sum of concurrencies of all scenarios in one particular second.

In the multi-test master, there are many scenarios, and there is no one second in which all of them reported their maximum number of VUs (this can be verified by looking at kpi.jtl file).

While in the combined master, there is only one scenario, so the above-described situation didn't happen (only one engine reports and sends samples to our Data service)."

The behavior you are seeing is as designed, and not a defect. Since there isn't any "one second" where all the scenarios report their maximum number of VUs back to the combined master, this is why you see a difference between the master versus each individual test.