Platform Health check error [VCFMS-HEALTH-002] due to missing VM template in vCenter during VCF Automation upgrade

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Symptoms

• VCF Automation upgrade fails during the package deployment phase.

• Listing the machines in the VCF Automation cluster using kubectl get machines -A , the output shows a machine in a Provisioning phase with an age of less than 10 minutes.

• The upgrade task in the Fleet Management fails with following health check error:

  Platform Health Check Error [platform-package-core : 1 of 1 resources are not ok: vmsp-platform: wrong resource state: InProgress - package deployment is in progress;] [VCFMS-HEALTH-002]

• The capv-controller-manager pod logs reveal a reconciler error indicating that the required VM template cannot be found:

  E0825 HH:MM:SS.####### 1 controller.go:316] "Reconciler error" err="failed to reconcile VM: unable to find template by name \"/[DATACENTER-NAME]/vm/[Path to vcf-services-platform-template-9.#.#.#.#######]\": vm '/[DATACENTER-NAME]/vm/[Path to vcf-services-platform-template-9.#.#.#.#######]' not found" controller="vspherevm" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="VSphereVM" VSphereVM="vmsp-platform/[AUTO-VM-NAME]" namespace="vmsp-platform" name="[AUTO-VM-NAME]" reconcileID="UUID"

Environment

VCF Automation 9.x

Cause

This issue occurs when the VCF Service Runtime Platform template is missing from the target location in vCenter. The capv-controller expects the template to be in the exact inventory path defined within the vmsp-platform package deployment configuration.

Resolution

To resolve this issue, query the expected template path from the cluster configuration and move the template to that location in vCenter.

1. SSH into the VCF Automation node using the vmware-system-user credentials.

2.  Elevate your privileges to the root user:

   sudo su -

3.  Retrieve the correct template folder path from the package deployment configuration by executing:

   kubectl get pd vmsp-platform -n vmsp-platform -o jsonpath='{.spec.values.provider.vsphere.templateFolder}'

4. Log in to your vCenter Server using an admin account.

5. Locate the template file referenced in the error logs and move it to the exact folder path returned in Step 3.

6. Retry the failed upgrade task from the Fleet Management UI.