Upgrade of VCF Operations for Networks 6.13 to VCF Operations for Networks 9.0 failed at Step2

Products

• VCF Operations for Networks

Issue/Introduction

• Upgrade of VCF Operations for Networks 6.13/6.14 to VCF Operations for Networks 9.0 failed at Step2
• Legacy VCF Operations for Networks 6.13 imported to VMWare Cloud Foundation via Fleet Management.
• Error message seen on Aria Operations for Networks GUI.
  
  Failed to execute sudo -u ubuntu /home/ubuntu/build-target/launcher/upgrade-services.sh /tmp/upgrade_infra-base_246946764e6uviuz.tar.gz /home/ubuntu/build-target/infra-base ubuntu /home/ubuntu/build-target/deployment infra-base 24694676 /home/ubuntu/build-target/infra-base >> /var/log/arkin/centralized_upgrade.log2>&1
  
  Refer to below screenshots:
  
   
  
   

Environment

• VCF Operations for Networks 6.13.0
• VCF Operations for Networks 6.14.0

Cause

Multiple snapshots on each of the VCF Operations for Networks 6.13 nodes were seen  and alerts on vCenter for the storage as well and errors seen on underlying storage is suspected to have caused this issue.

Resolution

To fix this perform below steps:

1. Revert to snapshots.
2. Ensure all services are healthy and running after snapshots revert and disk utilization is good as well on the VCF Operations for Networks platform and collector node 6.13/6.14 deployment.
3. From VCF9 we went ahead and deleted the failed import deployment.
   Refer to screenshot below:
   
   
   
   
4. Take a New snapshot for Existing VMware Aria Operations for Networks Instance deployment from Fleet management.
5. Trigger upgrade again after importing Aria Operations for Networks 6.13/6.14 to Fleet Management.
   
   Refer to below screenshot:
   
   
   
   
   Note:
   When you import a component into 9.0 , which is of a legacy version , most of the day-n actions wont be there. until you upgrade it to the 9.x version.
   One point to note is that the "Trigger Inventory Sync" option does not appear after importing Operations for Networks 6.14 into Fleet Manager. It is necessary to start the Plan Upgrade to enable Trigger Inventory Sync and then continue with the Upgrade.
   Refer to documentation Import an Existing VMware Aria Operations for Networks Instance Into VCF Operations and Upgrade to VCF Operations for Networks https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/deployment/upgrading-cloud-foundation/preparing-your-vcf-9-management-components/phase-4-upgrade-additional-management-components-in-vcf-9.html
6. Upgrade to VCF 9 Operations for Networks should now complete and no error message should be seen.
7. Once the upgrade to is completed and successfully login to VCF 9 GUI and validate all is looking good, trigger  a manual Inventory sync on Fleet Manager.
8. Delete the  Snapshots were taken before triggering an upgrade.

 

Additional Information

Supported legacy upgrade paths:

Legacy VCF Operations for Networks version 6.13.0 > VCF Operations for Networks 9.0
Legacy VCF Operations for Networks version 6.14.0 > VCF Operations for Networks 9.0