Pass/Fail configuration lost in JMeter performance test when JMX script uploaded to test through BlazeMeter Jenkins Plugin

Products

• BlazeMeter

Issue/Introduction

Using the BlazeMeter Jenkins plugin, a job in Jenkins is configured to upload the JMX script from a local repository to a BlazeMeter performance test with Failure criteria configured in the test.

The expected Failure criteria are not returned to Jenkins to properly change the ci_status of the job in Jenkins.

Environment

Release :  SAAS

Component : BLAZEMETER PERFORMANCE TESTING

Cause

When the Jenkins plugin overwrites the active JMX script of an existing Performance test which has Failure criteria configured, by design, the configured Failure criteria are removed from the test.

Resolution

The Jenkins plugin integration is working as designed.

A workaround that can be considered is to create a new Taurus YAML file to configure the test to upload and run the JMeter script and include a passfail module in the YAML file that can be used to set the ci_status of the Jenkins job.

It would be necessary to re-configure the Jenkins job to use the YAML file as the main file, and upload the JMeter script as well as any required CSV files required for the test as additional files.

Here is a very simple example of a Taurus YAML script that can be used to run a JMeter script on an On-Premise Private Location (OPL) which uses a CSV file which will also return Pass/Fail criteria to be used to properly set the Jenkins ci_status for the job:

execution: 
- scenario: <scenarioName>
  concurrency:
    cloud: 100
  ramp-up: 1m
  hold-for: 60m
  locations: 
    harbor-<harborID>: 1
  files:
  - <fileName>.csv

reporting:
- module: passfail
  run-at: cloud
  criteria:
  - subject: avg-rt
    label: '<label from test>'
    condition: '>'
    threshold: 100ms
    fail: true

scenarios:
  <scenarioName>:
    script: <script name>.jmx

Unfortunately, the passfail criteria available in Taurus is a subset of the criteria available when configured in a JMeter test in BlazeMeter.  The 
Pass/Fail Criteria article provides more information for configuring this module.