ERROR: "Node-Policy stage failed. Reason: nodepolicy spec is updated, waiting for reconcile"

Products

• VMware Telco Cloud Automation

Issue/Introduction

• Nodeconfig cert errors with:
  error : failed calling webhook "validator.nodeconfig.acm.vmware.com": failed to call webhook: Post "https://nodeconfigvalidator.tca-system.svc:443/validate-nodeconfig?timeout=5s": x509: certificate has expired or is not yet valid: current time 2025-05-06T15:36:10Z is after 2025-05-06T14:05:00Z. Node-Policy stage failed. Reason: nodepolicy spec is updated, waiting for reconcile.

• caBundle base64 string in the ValidatingWebhookConfiguration section of node_operator.yaml is showing as expired
• Management clusters are failing to configure addons post-deployment.

Environment

TCA 2.3, 3.2

Cause

TCA nodes have an expired nodeconfig certificate as of May 6th 2025. This affects not only systems still on prior version of TCA, but also those who have migrated to newer version of TCA without upgrading the management cluster.

Resolution

For TCA 3.2:

Apply the Patch Tool for TCA 3.2.0.1 KB.

For TCA 2.3:

1. Download the script with below command and transfer it to TCA-CP in /tmp directory with any sftp tool e.g. winscp.
   curl -kLO https://packages.broadcom.com/artifactory/tca-distro/kb/2.3.0/update-nodeconfig-cert
2. SSH to the TCA-CP with admin user and switch to /tmp directory where the script is transferred
3. Query the cluster list managed by current TCA-CP with below command
   bash update-nodeconfig-cert -q
4. Update certificate for management cluster and all workload clusters (both v1 and v2 clusters) that belong to it. If one of the workload clusters failed, the command below will continue to update next. If cluster certificate is not expired before 2027, will skip it and continue to the next cluster.
   bash update-nodeconfig-cert -m <replace the name with management cluster name>