NSX Edge Upgrade Fails at 1% with "Cant check signature: No Public Key, Failed to verify Bundle"
Article ID: 424887
Updated On:
Products
Issue/Introduction
During an NSX upgrade, an Edge node upgrade may stop around 1% and fail while validating the Node Upgrade Bundle (NUB).
In the Edge syslog, bundle integrity verification fails when GPG attempts to verify the *.bundle.sig signature using /root/.gnupg/pubring.kbx keyring.
| ####-##-##T##:##:## edge - - - - [NSX] [nsx@6876 comp="nsx-edge" subcomp="upgrade-bundle"] upgrade_bundle_helper: Failed to verify bundle: ['gpg', '--homedir', '/root/.gnupg', '--verify', '/tmp/tmphj4urd4w/VMware-NSX-edge-<version>.bundle.sig', '/tmp/VMware-NSX-edge-<version>.bundle'] returned 2: b"gpg: Signature made <Date> UTC\ngpg: using RSA key ngpg: Can't check signature: No public key\n" |
Environment
VMware NSX 4.x
Cause
The Edge node cannot verify the upgrade bundle signature because the required Public Key is missing, stale, or corrupted in the GPG keyring database:
- /root/.gnupg/pubring.kbx does not contain the public key that matches the RSA key referenced by the bundle signature, or the .gnupg directory/keyring is missing/invalid.
Normal Log during verifying bundle:
|
####-##-##T##:##:## edge - - - - [NSX] [nsx@6876 comp="nsx-edge" subcomp="upgrade-bundle"] upgrade_bundle_helper: Verifying bundle VMware-NSX-edge-<version>.bundle with signature VMware-NSX-edge-<version>.bundle.sig |
Problematic Flow:
| ####-##-##T##:##:## edge - - - - [NSX] [nsx@6876 comp="nsx-edge" subcomp="upgrade-bundle"] upgrade_bundle_helper: Verifying bundle VMware-NSX-edge-<version>.bundle with signature VMware-NSX-edge-<version>.bundle.sig ####-##-##T##:##:## edge - - - - [NSX] [nsx@6876 comp="nsx-edge" subcomp="upgrade-bundle"] upgrade_bundle_helper: Failed to verify bundle: ['gpg', '--homedir', '/root/.gnupg', '--verify', '/tmp/VMware-NSX-edge-<version>.sig', '/tmp/VMware-NSX-edge-<version>.bundle'] returned 2: b"gpg: Signature made <Datetime> UTC\ngpg: using RSA key \ngpg: Can't check signature: No public key\n" ####-##-##T##:##:## edge NSX 2241 - [nsx@6876 comp="nsx-edge" subcomp="upgrade-agent" tid="2369" level="ERROR" errorCode="MPA50007"] Error verifying nub signature http://<ip>/repository/<version>/Edge/nub/VMware-NSX-edge-<version>.nub, error msg: Checking upgrade bundle /var/vmware/nsx/file-store/VMware-NSX-edge-<version>.nub contents#012Verifying bundle VMware-NSX-edge-<version>.bundle with signature VMware-NSX-edge-<version>.bundle.sig#012Failed to verify bundle: ['gpg', '--homedir', '/root/.gnupg', '--verify', '/tmp/VMware-NSX-edge-<version>.bundle.sig', '/tmp/VMware-NSX-edge-<version>.bundle'] returned 2: b"gpg: Signature made <Date> UTC\ngpg: using RSA key \ngpg: Can't check signature: No public key\n"#012 |
Resolution
1. Copy pubring.kbx from a healthy node
If you already confirmed the healthy node upgrades successfully, you can copy its keybox and retry (as you did).
## ensure the source node is the same NSX major/minor/build family and keep backups.
2. Rebuild keyring by re-importing NSX public keys
1) Login to the failed manager as root
2) Import public keys again:
|
example: gpg --homedir /root/.gnupg --import /opt/vmware/nsx-node-api/etc/publickey_530C79E6.asc |
3) Verify keys are present
| gpg --list-keys --homedir /root/.gnupg/ ls -l /root/.gnupg/ |
4) Retry the Edge upgrade
Additional Information
Another possible cause:
https://knowledge.broadcom.com/external/article?articleNumber=369092
Feedback
