vSphere with Tanzu Supervisor operations failing with custom VC certificates with key size greater than 8K

Products

• VMware vCenter Server

Issue/Introduction

Document steps to recover from supervisor enablement or upgrade failure when VC certificate key size is greater than 8192 bits.

Symptoms:

With vSphere 8.0 Update 2b, the maximum key size of a CSR in a vCenter system is down to 8192 bits from 16384 bits.

1. Supervisor enablement fails with VC certificate key size greater than 8192 bits.
2. Supervisor upgrade gets stuck at 3% with VC certificate key size is greater than 8192 bits.
3. Supervisor components like NCP/CSI are entered into CrashLoopBackOff when VC certificate key size is greater than 8192 bits.

Environment

VMware vCenter Server 8.0.2

Cause

Golang reduced the supported key size to a max of 8192 bits in order to address performance issue. Since many of vSphere with Tanzu components are in Golang, it is impacted.

Resolution

Currently there is no resolution.

Workaround:

Regenerate any VC certificate that has a key size greater than 8192 bits

Follow the below Steps:

1. Identify any certificates with key size greater than 8192 bits for store in TRUSTED_ROOTS MACHINE_SSL_CERT vpxd-extension wcp ; do echo $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store "$store" --text | grep Public-Key; done
2. Replace any certificates whose key size is greater than 8192 bits.
3. Re-register VC in NSX if using NSX.
4. Restart WCP service, refer to 96473.

Additional Information

Impact/Risks:

With vSphere 8.0 Update 2b, the maximum key size of a CSR in a vCenter system is down to 8192 bits from 16384 bits.