Error: Node version mismatch during VCF 9.1 upgrade - VCF Operations for Networks

Products

• VCF Operations for Networks

Issue/Introduction

• The VMware Cloud Foundation (VCF) 9.1 upgrade for the management vCenter stalls due to a Node version mismatch error within VCF Operations for Networks.
• Error message see is:
  Node Version Mismatch detected - Version for Platform<##.##.###.##> is 9.1.0.0.25318550, Version for Proxy<##.##.###.##> is 9.0.2.0.25119537
• This occurs when platform nodes are upgraded to VCF 9.1 General Availability (GA), but a collector node is either offline or replaced with a version running Express Patch 1 binaries, preventing cluster synchronization.
  
• VCF Operations administrative interface fails to display active collectors.
• Synchronization and inter-node communication failures occur within the cluster.
• The patch version manifest (cat build-target/deployment/patch.txt) on the platform and collector node indicates build version 9.1.0.0100.25427421 (Express Patch 1), while platform nodes are on the 9.1 GA release.
  See screenshot showing the error on the collector node.
  

 

Environment

• VMware Cloud Foundation 9.1
• VCF Operations for Networks 9.1

Cause

• The issue is caused by a software version mismatch between the clustered platform nodes (9.1 GA) and the collector node.
• This typically happens when a replacement collector is deployed using a newer binary source (such as Express Patch 1) than the existing GA cluster.
• VCF Operations for Networks requires strict version parity; the platform nodes will reject heartbeats from any node running a different build version.

Resolution

Follow the workaround steps to align the node versions and proceed with the upgrade:

1. Delete the collector node that was incorrectly deployed with VCF 9.1 Express Patch (EP1) binaries.
2. Power on the original collector node that was at version 9.0.2.
3. Re-IP the collector node to ensure it utilizes the correct, designated IP address.
4. Navigate to the VCF Operations UI and initiate the upgrade to the 9.1 GA version.
5. Monitor the process for approximately 30 minutes until the status shows as healthy and the version mismatch alert clears.

Additional Information

For defects and enhancements, subscribe to this article (Reference: https://knowledge.broadcom.com/external/article/275360) to be updated on fix status.

To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.