Async patch tool download bundles fails with error : One or more operationsmanager service workflows are in progress

Products

• VMware SDDC Manager

Issue/Introduction

• Async Patch Tool download bundles fails and the following is seen in async_patch_tool.log :

[ERROR] Precheck failed due to the following errors: 

[ERROR] FOUND IN PROGRESS SDDC SERVICE WORKFLOWS
Error Message: Found one or more sddc service workflows in-progress on the system. Remedy: Please make sure there are no in-progress SDDC service workflows on the system. 

INFO Updating telemetry details of operation: ENABLE_ASYNC_PATCH:precheck
INFO Added operation ENABLE ASYNC PATCH:precheck and its status FAILURE to async patch tool metrics
INFO Successfully updated telemetry details of operation: ENABLE_ASYNC_PATCH:precheck
[ERROR] Error while initiating required functionality: ENABLE_ASYNC_PATCH

[ERROR] ENABLE PATCH PRECHECK FAILEDINFO
Error Message: Enable Patch pre-checks failed due to above errors.
Remedy: Please resolve all above errors and rerun the AP tool to enable patches. Error Stacktrace:

Copying log file for SOS collection to /var/log/vmware/vcf/lcm/tools/asyncpatchtool Exception in thread "main" java.lang.reflect.InvocationTargetException
[main] ERROR [com.vmware.evo.sddc.lcm.tools.common.WorkflowStatusValidationUtil]
                      One or more operationsmanager service workflows are in progress. In Progress Workflows- [Assess SDDC Domain(s), Updating licenses for resources part of domain {domain_id}]
PrecheckTask =
{
        Name = NO_IN_PROGRESS_SDDC_SERVICE_WORKFLOW_VALIDATION,
        Start Time = YYYY-MM-DD 00:00:00.000,
        End Time = YYYY-MM-DD 00:00:00.000,
        PrecheckError = [PrecheckError(errorCode=FOUND_IN_PROGRESS_SDDC_SERVICE_WORKFLOWS, errorDescription=Found one or more sddc service workflows in-progress on the system., errorType=ERROR, metadata=null, impact=null, remediation=Please make sure there are no in-progress SDDC service workflows on the system.)],
        Health Status = RED
}],
        Aggregated Health Status = RED,
        Required SDDC bundles = [],
        Are Precheck Tasks All Completed = true
}

• Similar is seen in operationsmanager.log:

DEBUG [vef_cm, 0000000000000000, 0000] [c.v.e.s.c.a.WorkflowHelperUtil, main] Getting executionID of all in-progress workflows
DEBUG [vef_cm, 0000000000000000,0000] [c.v.e.s.c.s.WorkflowHelperUtil, main] Found an in-progress workflow with executionID {execution_id}
DEBUG (vcf_cm, 0000000000000000,0000] [c.v.e.s.c.s.WorkflowHelperUtil, main] Found an in-progress workflow with executionID {execution_id}
ERROR [vcf_cm, 0000000000000000,0000] [c.v.v.h.c.e.CommDecommTasksCleaner, main] An error occurred while updating in-progress tasks state upon OperationsManager restart java.lang.NullPointerException: Cannot invoke "String.equals (Object)" because "executionName" is null
at com.vmware.vcf.hostmanager.comm.execution.CommDecommTasksCleaner.cleanUpInProgressTasks (CommDecommTasksCleaner.java:41)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Cause

The internal API call failed as there is a workflow with null name.

Stale workflow still present in the SDDC Manager DB.

Resolution

Please engage with Broadcom Support if you are facing this issue, referencing this KB article.