Updating vCenter fails with error "Downloading RPM vsphere-ui-7.0.3.00300-9405520.noarch.rpm"

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Logging in to the vCenter Appliance Management Interface (VAMI) using root credentials returns an error "Downloading RPM vsphere-ui-7.0.3.00300-9405520.noarch.rpm" post entering the credentials.

 

    

 

• Similar entries may exist in var/log/vmware/applmgmt/update_microservice.log

 

YYYY-MM-DD HH-MM-SS,182 - 7353 -    integrity_checker::                 log_and_print:  79 -     INFO - Public key: /var/vmware/applmgmt/fileintegrity/pub.key is missing, generating one
YYYY-MM-DD HH-MM-SS,247 - 7353 -    integrity_checker::                 log_and_print:  75 -    ERROR - ERROR: Exception while verifying signature. {
"detail": [
{
"id": "install.ciscommon.command.errinvoke",
"translatable": "An error occurred while invoking external command : '%(0)s'",
"args": [
"Stderr: "
],
"localized": "An error occurred while invoking external command : 'Stderr: '"
}
],
"componentKey": null,
"problemId": null,
"resolution": null
}
YYYY-MM-DD HH-MM-SS,248 - 7353 -    integrity_checker::                 log_and_print:  75 -    ERROR - /etc/vmware/appliance/fileintegrity_config.json config file verification failed
YYYY-MM-DD HH-MM-SS,251 - 7353 -  update_microservice::                     _runEvent: 417 -    ERROR - Event callback failed: Exception('Failed verify config file') 'Traceback (most recent call last):\n  File "/usr/lib/applmgmt/update/py/vmware/appliance/update/update_microservice.py", line 410, in _runEvent\n    coroutine.send(1)\n  File "/usr/lib/applmgmt/update/py/vmware/appliance/update/update_microservice.py", line 176, in stageEventHandler\n    coroutine.send(i)\n  File "/usr/lib/applmgmt/update/py/vmware/appliance/update/update_b2b.py", line 1261, in stage\n    integrity_check_enabled = integrity_check.is_integrity_check_enabled()\n  File "/usr/lib/vmware/site-packages/cis/integrity_checker.py", line 730, in is_integrity_check_enabled\n    raise Exception("Failed verify config file")\nException: Failed verify config file\n'

 

• To verify the error, ssh to the VCSA and run the command:

openssl dgst -verify /var/vmware/applmgmt/fileintegrity/pub.key -signature /var/vmware/applmgmt/fileintegrity/fileintegrity_config.sig /etc/vmware/appliance/fileintegrity_config.json

The command should return a response "Verification Failure".

 

Environment

VMware vCenter Server 7.0.x

Resolution

This issue is resolved in VMware vCenter Server 7.0 U3c. However,  it's advised to update to the latest vCenter 7.0 release. Find the vCenter Server download packages by doing the following:

1. Login to the Broadcom Support Portal
2. Select "My Downloads"
3. Select "VMware vCenter Server"
4. Select the major version, i.e.
   "VMware vCenter Server 7.x"
5. Select the most recent release, such as, "7.0U3p"

Workaround:

To workaround this issue, please follow these steps:

1. Take an offline snapshot (with VCSA powered off),
2. Login to VCSA through ssh using putty.
3. Download the script generate_signature.py from the attachment section to in the article.
4. Upload the script to the VCSA " root directory" using WINSCP

Note: If there is an error while trying to login to VCSA through WINSCP , run the below command on VCSA (SSH):
# chsh -s /bin/bash root

5. Run the script using the command:

# python generate_signature.py

4. Run the command:

# openssl dgst -verify /var/vmware/applmgmt/fileintegrity/pub.key -signature /var/vmware/applmgmt/fileintegrity/fileintegrity_config.sig /etc/vmware/appliance/fileintegrity_config.json

This should return a "Verified OK" response.

5. Run the following commands:

# service-control --stop applmgmt
# rm -rf /storage/core/software-update/*
# rm -rf /storage/db/patching.db
# mv /storage/core/software-packages/staged-configuration.json /storage/core
# mv /etc/applmgmt/appliance/software_update_state.conf /storage/core
# service-control --start applmgmt

6. Retry the update.