Datastore name change in VC triggers SSPi 5.1 upgrade failure with "TimeoutError: Rollout of kubeadmControlPlane <control-node name> did not complete within 2700 seconds" ERROR

Products

VMware vDefend Firewall VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

SSP-I upgrades can fail for multiple reasons. When the following error appears and the first control plane node is not created in vCenter (and no IP address is assigned), it usually indicates a failure early in the machine provisioning process.

Error Message

An unexpected exception occurred: TimeoutError: Rollout of kubeadmControlPlane <control-node name> did not complete within 2700 seconds

This error occurs after the SSP-I upgrade pre-check stage, with no specific cluster health issues indicated, while the system waits for the first control plane node to be provisioned. The provisioning loop typically repeats every 5–7 minutes and eventually fails with the error above.

Verifying the Issue

Run the following command:

kubectl get machines -A

Example output:

NAMESPACE   NAME                    CLUSTER   NODENAME                PROVIDERID                                       PHASE         AGE   VERSION
ssp2        ssp2-8r44z              ssp2      ssp2-8r44z              vsphere://423feb19-8ae4-f695-9e42-b616f20f76a2   Running       15d   v1.33.3
ssp2        ssp2-md-0-gmprs-7nbnp   ssp2      ssp2-md-0-gmprs-7nbnp   vsphere://423fdf6f-7298-0739-e1d9-24bb6ba49779   Running       20d   v1.33.3
ssp2        ssp2-md-0-gmprs-gwfpf   ssp2      ssp2-md-0-gmprs-gwfpf   vsphere://423fd133-246a-14df-8c17-1bdf1609e09a   Running       20d   v1.33.3
ssp2        ssp2-md-0-gmprs-nhcnp   ssp2      ssp2-md-0-gmprs-nhcnp   vsphere://423fe492-e2ba-e6d6-95df-60881869e548   Running       20d   v1.33.3
ssp2        ssp2-md-0-gmprs-qfth6   ssp2      ssp2-md-0-gmprs-qfth6   vsphere://423f8ec4-add9-c56a-8661-2b3157aaa87e   Running       20d   v1.33.3
ssp2        ssp2-md-0-gmprs-qvx7j   ssp2      ssp2-md-0-gmprs-qvx7j   vsphere://423fd128-8007-41e0-0a46-a41e9d390fad   Running       20d   v1.33.3
ssp2        ssp2-nzf2m              ssp2      ssp2-nzf2m              vsphere://423fd236-0855-f659-594a-8cbda7eab944   Running       15d   v1.33.3
ssp2        ssp2-t2ft2              ssp2      ssp2-t2ft2              vsphere://423f757d-be49-017c-5140-a2ef0d897c6e   Running       15d   v1.33.3
ssp2        ssp2-vxf8s              ssp2      ssp2-vxf8s              Provisioning

The machine description for the failed entry will typically show additional details confirming the provisioning problem.

kubectl describe machine <problem-machine-name> -n <name space or cluster name> 
...
  Conditions:
    Last Transition Time:  2025-10-27T16:54:40Z
    Message:               1 of 2 completed
    Reason:                CloningFailed
...
    Type:                  BootstrapReady
    Last Transition Time:  2025-10-27T16:54:40Z
    Message:               unable to get datastore /Production/datastore/General Use Datastores/Mgmt/mgmt_f_vm_xxxxx_01 for "infrastructure.cluster.x-k8s.io/v1beta1, Kind=VSphereVM <node-name>/<control-node name>": datastore '/Production/datastore/General Use Datastores/Mgmt/mgmt_f_vm_xxxxx_01' not found

Environment

Security Services Platform Installer(SSP-I) - 5.1

Cause

When a machine is being provisioned, the clone operation begins from the Content Library.

In SSP 5.0, the system stored Content Library object names as identifiers in the internal database, including fields such as image name and datastore path.

If these names or paths change later, SSP 5.0 to 5.1 upgrade may fail with “path not found” or “file not found” errors.

The issue occurs because the datastore path changed after the initial SSP-I 5.0 greenfield deployment.

The Content Library is only used when the Kubernetes cluster needs to provision a new machine, such as during new pod deployment or a rolling restart.

When these events occur, the clone operation fails because the OVA image cannot be found at the updated datastore location.

Key points:

- - No issues can be seen with the storage policy.
  - The datastore path for the Content Library had changed.
  - The problem only appears during machine provisioning.
  - Pod restarts do not expose the issue because they do not trigger cloning from the Content Library.

Resolution

Correcting the Datastore Path

Identify the correct datastore path referenced in the logs:

datastore '/Production/datastore/General Use Datastores/Mgmt/mgmt_f_vm_xxxxx_01

Check the actual datastore path in vCenter to confirm whether it matches the original value.

Example

Current vCenter path

datastore '/Production/datastore/General Use Datastores/Mgmt Datastores/mgmt_f_vm_xxxxx_01

Original datastore path (expected)

datastore '/Production/datastore/General Use Datastores/Mgmt/mgmt_f_vm_xxxxx_01

Update or revert the datastore name/path in vCenter so that it matches the original value recorded in the configuration and logs.

In the vshpere client, navigate to the datastore.
Right-click the datastore to rename, and select Rename.
Enter a new datastore name or path. See Basic Administrative Operations for vSphere Datastores

This ensures that the clone and provisioning processes can correctly locate the OVA image that was used during initial deployment.