Upgrade Failure of VCF Automation and VMware Identity Broker from 9.0.0.0 to 9.0.1.0 with Error "LCMVMSP10037"
search cancel

Upgrade Failure of VCF Automation and VMware Identity Broker from 9.0.0.0 to 9.0.1.0 with Error "LCMVMSP10037"

book

Article ID: 417145

calendar_today

Updated On:

Products

VCF Automation

Issue/Introduction

During the upgrade of VMware Cloud Foundation (VCF) Automation or VMware Identity Broker from version 9.0.0.0 to 9.0.1.0, the upgrade fails on Fleet Management with the error LCMVMSP10037.

The following error can be seen: /var/log/vrlcm/vmware_vrlcm.log)

INFO vrlcm[756715] [http-nio-8080-exec-6] [c.v.v.l.r.c.RequestController] – Retry error cause data : [ {
"messageId" : "LCMVMSP10037",
"message" : "Applying patch on the application platform failed.",
"retry" : true,
"exceptionMessage" : "Product deployment failed.",
"exceptionStackTrace" : "com.vmware.vrealize.lcm.vmsp.common.exception.VmspPatchingException: No failed pods found.\n\nNo failed pods found.\n\n\tat com.vmware.vrealize.lcm.vmsp.plugin.tasks.VmspPatchApplicationPlatformTask.execute(VmspPatchApplicationPlatformTask.java:81)\n\tat com.vmware.vrealize.lcm.automata.core.TaskThread.run(TaskThread.java:62)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n",
"localizedMessageId" : "LCM_VMSP_PKG_PUSH_ERROR_0001",
"parameters" : [ "No failed pods found.\n\nNo failed pods found.\n" ],

 

Additionally, when checking the bundle status in VCF Automation or VMware Identity Broker, the following failure is observed:

root@ [ ~ ]# kubectl get bundle -A
NAMESPACE       NAME   AGE   PHASE    STATUS
vmsp-platform   vmsp   66m   Failed   failed to import /tmp/vmsp-pkg3021982879/vmsp-photon-kube-v1.32.0/ovas/cap-poi/vcf-services-platform-template-9.0.1.0.24940697.ova: network '/datacenter/network/network-name' not found


OR 

root [~]# kubectl get bundle -A
NAMESPACE       NAME   AGE    PHASE    STATUS
vmsp-platform   vmsp   5m1s   Failed   failed to get the datastore: datastore '/datacenter/datastore/datastore-name' not found

Environment

  • VCF Automation 9.x
  • VMware Identity Broker 9.x

Cause

The upgrade fails because the vSphere network or datastore configured in the vmsp-platform Provider Definition (PD) does not exist or has been renamed in vCenter. As a result, the OVA import fails with the error “network not found” or "datastore not found".

Resolution

Step 1: Verify Datastore and Network Location

  1. Log in to the vCenter Server and ensure that the datastore and network assigned to the VCF Automation components are located at the root of the Datacenter or within the exact path specified in the Provider Definition.
  2. If objects were moved into folders, move them back to the root or update the PD path in Step 4.

Step 2: Launch Debug Pod

  1. Start a temporary debugging session in the vmsp-platform namespace:
    kubectl run -n vmsp-platform --restart=Never --rm -i --tty debug --image=registry.vmsp-platform.svc.cluster.local:5000/images/debug:v1.0.7 --command -- bash

Note: This command launches a temporary debugging pod with an interactive shell for network verification or connectivity testing.

Step 3: Authenticate and Verify Objects

  1. Set the environment variables. Use single quotes for the password to handle special characters.
    export GOVC_URL='<vCenter-FQDN>' export GOVC_USERNAME='[email protected]' export GOVC_PASSWORD='####' export GOVC_INSECURE=1
  2. List available network objects. If using a machine certificate from an external authority, include the -k parameter: 
    govc find -k -type n .
  3. List available datastore objects:
    govc datastore.info
  4. Compare the output to the paths configured in the system.

NOTE: if any folder paths are missing from the configuration.

Step 4: Update Provider Definition (If paths changed)
If the paths discovered in Step 3 differ from the current configuration, update the vmsp-platform Provider Definition:

  1. Run the command below: 
    kubectl edit pd -n vmsp-platform vmsp-platform
  2. Locate the following
     spec: values: provider: vsphere.
  3. Update the network or datastore fields with the correct paths.

Step 5: Refresh Inventory and Retry

  1. In the VCF Operations UI, navigate to the component and trigger a manual inventory sync.
  2. Monitor the PD status until it shows:
    Successful: kubectl get pd -n vmsp-platform vmsp-platform
  3. Restart the upgrade from Fleet Management.

Additional Information

  • The error LCMVMSP10037 indicates a patching failure caused by missing or invalid vSphere configuration.
  • The govc find command helps verify the actual vSphere network path.
  • Always ensure vCenter inventory objects referenced in PD (network, datastore, cluster) are not renamed or removed before an upgrade
  • To be updated on the fix status of this issue,Subscribe to a Broadcom knowledge article by article or product
Powered by Wolken Software