[VMC] VCGA is failing to update

Products

• VMware Cloud on AWS
• VMware Cloud on Dell EMC

Issue/Introduction

Impact/Risks:
The VCGA will continue to fail until the resolution is completed.

Provide steps to force the update of the VCGA.

Symptoms:

Customer's VCGA is attempting to update to the latest version (8.0.0.10000), but is failing.

Example Screenshot:


Example Log entries:

2023-07-27T13:40:11.274 [34128]DEBUG:utils.update_utils:Version file not modified, will use cached local version 8.0.0.00000 for Platform'

2023-07-26 21:10:20,336 - 56320 -    update_b2b_target::               b2bRequirements: 433 -    DEBUG - Done with B2Brequirements with script hook result <vmware.update.specs.RequirementsResult object at 0x7fbf1d107210> and selected disk partition /storage/core
2023-07-26 21:10:20,336 - 56320 -  task_manager_target::                        update:  80 -    DEBUG - UpdateTask: status=FAILED, progress=0, message={'id': 'com.vmware.appliance.nothing to stage', 'default_message': 'All the packages are already installed.', 'args': []}
2023-07-26 21:10:20,338 - 56320 -    update_b2b_target::                         stage:1153 -    DEBUG - Returning from stage(), all packages are already installed
2023-07-26 21:10:20,338 - 56320 -   dbfunctions_target::                    executeDML:  54 -    DEBUG - Executing Query {INSERT INTO progress VALUES(NULL, ?,         DATETIME('now'), ?, ?, ?)} with parameters ('download', 'All RPMs already staged', 100, 40)
2023-07-26 21:10:20,340 - 56320 -  update_microservice::                     _runEvent: 420 -    ERROR - Event callback failed: UnboundLocalError("local variable 'stageDir' referenced before assignment") 'Traceback (most recent call last):\n  File "/storage/core/software-update/updates/8.0.0.10000/update/update_microservice.py", line 413, in _runEvent\n    coroutine.send(1)\n  File "/storage/core/software-update/updates/8.0.0.10000/update/update_microservice.py", line 253, in stageAndInstallEventHandler\n    coroutine.send(i)\n  File "/storage/core/software-update/updates/8.0.0.10000/update/update_b2b_target.py", line 1215, in stage\n    _cleanup([path.dirname(stageDir), STAGE_DIR, WORK_UPDATES],\nUnboundLocalError: local variable \'stageDir\' referenced before assignment\n'

sk4@blr-dbc2103:/dbc/blr-dbc2103/sk4/LOGS/Triage/VMC-Gateway/vc-vmware-vcgw-prod.intranet.local-2023-07-27--13.56-40887/etc/applmgmt/appliance$ cat software_update_state.conf 

{

    "state": "INSTALL_FAILED",

    "version": "8.0.0.10000",

    "latest_query_time": "2023-07-26T21:10:11Z",

    "operation_id": "/storage/core/software-update/stage_install_operation"

 

Cause

The issue is not due to any patching code, but rather a cleanup of stage files is required.

Resolution

1. Confirm the steps in this KB have been completed prior to proceeding with the step 2 - Alert on Cloud Gateway Appliance- "Cloud Gateway Update Failed: Contact support"

2. If the above KB does not remediate the issue, please proceed with creating a Support Request with VMC GS (Creating and managing Broadcom support cases) with an OnPrem vCenter Log Bundle, VCGA Log Bundle, a screenshot of the failed update from the VCGA VAMI and this KB attached.