Using the BlazeMeter Jenkins plugin, a job in Jenkins is configured to upload the JMX script from a local repository to a BlazeMeter performance test with Failure criteria configured in the test.
The expected Failure criteria are not returned to Jenkins to properly change the ci_status of the job in Jenkins.
Release : SAAS
Component : BLAZEMETER PERFORMANCE TESTING
When the Jenkins plugin overwrites the active JMX script of an existing Performance test which has Failure criteria configured, by design, the configured Failure criteria are removed from the test.
The Jenkins plugin integration is working as designed.
A workaround that can be considered is to create a new Taurus YAML file to configure the test to upload and run the JMeter script and include a passfail module in the YAML file that can be used to set the ci_status of the Jenkins job.
It would be necessary to re-configure the Jenkins job to use the YAML file as the main file, and upload the JMeter script as well as any required CSV files required for the test as additional files.
Here is a very simple example of a Taurus YAML script that can be used to run a JMeter script on an On-Premise Private Location (OPL) which uses a CSV file which will also return Pass/Fail criteria to be used to properly set the Jenkins ci_status for the job:
execution:
- scenario: <scenarioName>
concurrency:
cloud: 100
ramp-up: 1m
hold-for: 60m
locations:
harbor-<harborID>: 1
files:
- <fileName>.csv
reporting:
- module: passfail
run-at: cloud
criteria:
- subject: avg-rt
label: '<label from test>'
condition: '>'
threshold: 100ms
fail: true
scenarios:
<scenarioName>:
script: <script name>.jmx
Unfortunately, the passfail criteria available in Taurus is a subset of the criteria available when configured in a JMeter test in BlazeMeter. The
Pass/Fail Criteria article provides more information for configuring this module.