DevTest Fail Test Case Companion is Not Working as Expected
Article ID: 115227
Updated On:
Products
Issue/Introduction
Using Fail test case companion to fail the test case when any one of the assertions fails, but it is not working as expected. It still shows all test cases as PASSED in DevTest Portal.
Environment
All supported DevTest releases.
Cause
N/A
Resolution
This is an expected behavior.
If a step in a test case has an error, the Fail Test Case companion marks a test case as failed, even if it ends successfully. The companion registers an event listener for the EVENT_TRANSFAILED event.
Example: A WSDL validation assertion on a web service step has an error. You want to be informed that the validation failed, but you also want the test case to continue.
This companion has no configuration parameters.
The status "PASSED" for a test step means that the step has executed successfully but it doesn't mean that the functionality is passed or failed. DevTest records whether or not a given step has passed or failed, but this may not be an indicator of the business reason the step failed.
Whenever there is an assert to fail, the test case will stop running immediately and then end in a fail state. But when an assert is to error rather than fail, the specific step will be marked with error, the rest of the assertions and steps will keep running. The test will show it had "errors" but end in a PASS. However, if the companion "Fail test case companion" is applied and if there are any errors in the run, the test will be marked as FAILED at the end. Generally in case of Fail test case companion, "error" means DevTest is failing the test case but will continue with execution.
Ideally, this is correct because generally if a test step fails, DevTest will fail the test case and does not execute the next step. For example, if the first step of a test case is to login to an application and if it failed, DevTest will fail the test case and stop the execution.
Attachments
Feedback
