No Key matching certificate found. Ensure that the certificate matches certificate signing request and that it did not expired
Article ID: 332573
Updated On:
Products
Issue/Introduction
Symptoms:
When we try to extend the Pod CIDRs on the WCP-enabled cluster, we get an error as: No Key matching certificate found. Ensure that the certificate matches the certificate signing request and that it did not expire
WCP Logs from the VCSA shows below log details:
2020-08-10T11:51:49.655Z debug wcp [opID=5f2f9609] Updating new configuration for cluster domain-c8. UpdateSpec {SizeHint:<nil> NetworkProvider:<*>(0xc0012e0c30)NSXT_CONTAINER_PLUGIN NcpClusterNetworkSpec:<*>(0xc002e14a00){PodCidrs:[{Add
ress:192.168.202.0 Prefix:23} {Address:192.168.208.0 Prefix:23}] IngressCidrs:[{Address:10.227.3.128 Prefix:25}] EgressCidrs:[{Address:10.227.2.128 Prefix:25}] DefaultIngressTlsCertificate:<nil>} MasterDNS:<nil> WorkerDNS:[172.17.80.3] M
asterDNSSearchDomains:<nil> MasterNTPServers:<nil> MasterStoragePolicy:<nil> EphemeralStoragePolicy:<nil> LoginBanner:<nil> ImageStorage:<nil> DefaultImageRegistry:<nil> DefaultImageRepository:<nil> TlsEndpointCertificate:<nil> DefaultKu
bernetesServiceContentLibrary:<nil>}
2020-08-10T11:51:49.661Z debug wcp [opID=5f2f9609] Getting private key for certificate: -----BEGIN CERTIFICATE-----
MIID5TCCAs2gAwIBAgIJAO4Nq5vvaZ77MA0GCSqGSIb3DQEBCwUAMIGhMQswCQYD
VQQDDAJDQTEXMBUGCgmSJomT8ixkARkWB3ZzcGhlcmUxFTATBgoJkiaJk/IsZAEZ
FgVsb2NhbDELMAkGA1UEBhMCVVMxEzARBgNVBAgMCkNhbGlmb3JuaWExIzAhBgNV
BAoMGnNvZ3JhbWlocXZtY2MwMi5pbmZvNTMuY29tMRswGQYDVQQLDBJWTXdhcmUg
RW5naW5lZXJpbmcwHhcNMjAwNjMwMTMyOTQzWhcNMjIwNjMwMTMyOTQzWjBEMSMw
IQYDVQQLExpzb2dyYW1paHF2bWNjMDIuaW5mbzUzLmNvbTEdMBsGA1UEAxMUZGVm
YXVsdGluZ3Jlc3MubG9jYWwwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDIUD7bqsUxI68WRTVBReJWSS5DvMK0uApabwCmiGqvnkN/VqnfZ+mDY/6AiQ76
Djv9yFBDK59wSrp7bcYLkWwb6jXK+MPrn5FvVYG0DXiaSkMXlkoNmPu2Auoa+j0T
V96debtiCDDPA5ovfJlUfkPOWK2CTDPOboZrD+Gis927uzjv9ctiYLgi9u/Q4/NN
aC8b6ldpvPVyIAewm9c5TcH3ouUhw7Fx+KqArJAUlPxkn43TO0ADjyHPyCmSa7ip
dt1Qqf+0etiAtFeWnAGTzuBQGppP9sm9nZTYFze2dTZxDRJzIfWcI07rioqIc5+T
HNbD9DsqH4V+qtGo0Np4+jsnAgMBAAGjfDB6MAsGA1UdEQQEMAKBADAfBgNVHSME
GDAWgBSATXO0Xl0tWjEYryMWT/1sH0LFIDBKBggrBgEFBQcBAQQ+MDwwOgYIKwYB
BQUHMAKGLmh0dHBzOi8vc29ncmFtaWhxdm1jYzAyLmluZm81My5jb20vYWZkL3Zl
Y3MvY2EwDQYJKoZIhvcNAQELBQADggEBAHhzMIA0txsFxoTgTW12VqwHtNp3Z7Gy
cD0hHnd0pKJ/NKzl1DGgjsptN7MyO1QAsYf6QGS6NQTXdRB00AC+c23OgHXOufk1
jT15mtGbLfAfN3oGUrf5ohy98Erz1td61HTxO0ihMPacyjbyiVyglhNHHSrwISyp
A8WgWeQ8rkvo4Omky6HtQf56mm0mcvEEO3ZNkpTMAHDxTfrzi0idRUisgqlyJwvR
sf5SrcZ8ST0Wn65ElFYHjvHGYotPFRXGrLZL2gu9IfDmeFEHxmTqKJpTfZ3krSRu
/Q+0Ci9WdghSsgsfVOuMeTXc2CqPatbtUmlsd+PUXix4EVBg1Y7TwVc=
-----END CERTIFICATE-----
2020-08-10T11:51:49.662Z error wcp [opID=5f2f9609] Failed to retrieve private key matching the certificate: com.vmware.vapi.std.errors.not_found
2020-08-10T11:51:49.662Z error wcp [opID=5f2f9609] Error updating new configuration for cluster domain-c8. Err com.vmware.vapi.std.errors.not_found
2020-08-10T11:51:49.662Z debug wcp [opID=vapi] Validating output
2020-08-10T11:51:49.662Z debug wcp [opID=vapi] Sending response with output {"error":{"ERROR":{"com.vmware.vapi.std.errors.not_found":{"data":{"OPTIONAL":null},"error_type":{"OPTIONAL":null},"messages":[{"STRUCTURE":{"com.vmware.vapi.std.localizable_message":{"args":[],"default_message":"No key matching certificate found. Ensure that the certificate matches certificate signing request and that it did not expire.","id":"vcenter.wcp.cluster.tls.csrnotfound","localized":{"OPTIONAL":"No key matching certificate found. Ensure that the certificate matches certificate signing request and that it did not expire."},"params":{"OPTIONAL":null}}}}]}}}}
2020-08-10T11:51:50.071Z debug wcp informer.processLoop() lister.List() returned
2020-08-10T11:51:52.334Z debug wcp [opID=5f2f95be] No notifications. seqNum: 161, Current seqNum: 160
2020-08-10T11:51:56.518Z debug wcp [opID=5f2b062e] Got cached machine ID: 4366cbcc-3425-####-####-########599
2020-08-10T11:51:56.518Z debug wcp [opID=5f2b062e] Service account client already exists, reuse it.
2020-08-10T11:51:56.606Z info wcp [opID=5f2b062e] Reset service account password; user: wcp-appplatform-user-domain-c8-4366cbcc-3425-####-####-########599
2020-08-10T11:51:56.608Z info wcp [opID=5f2b062e] Successfully stored new service account password and time stamp to db for account wcp-appplatform-user-domain-c8-4366cbcc-3425-####-####-########599.
2020-08-10T11:51:56.662Z debug wcp [opID=5f2b062e] Secret vmware-system-appplatform-vc-auth already exists, trying to update it...
2020-08-10T11:51:56.664Z info wcp [opID=5f2b062e] Cannot find secret vmware-system-appplatform-vc-auth in vmware-system-appplatform-operator-system namespace. Error: secrets "vmware-system-appplatform-vc-auth" not found
2020-08-10T11:51:56.664Z error wcp [opID=5f2b062e] Failed to update secret vmware-system-appplatform-vc-auth in vmware-system-appplatform-operator-system namespace after resetting account wcp-appplatform-user-domain-c8-4366cbcc-3425-4a64-ad1e-8f12c8e92599 password. Err Kubernetes API call failed. Details secrets "vmware-system-appplatform-vc-auth" not found
2020-08-10T11:51:56.664Z error wcp [opID=5f2b062e] Failed to reset service account password for 2. Err Kubernetes API call failed. Details secrets "vmware-system-appplatform-vc-auth" not found. Retrying.
2020-08-10T11:52:00.089Z debug wcp informer.processLoop() lister.List() returned
Environment
Cause
Resolution
This will fixed in the future versions of WCP update.
Workaround:
NCP cluster ingress certificate needs to be updated which could be done via the UI.
1. In VC UI -> Select the affected WCP Cluster
2. Click "Configure".
3. On the left hand side, under "Namespaces", click "Certificates".
4. For the "NSX Load Balancer" certificate, click "Actions" and then "Generate CSR".
5. Download the CSR provided.
6. Sign the CSR with any desired certificate authority. To use VMCA, ssh into vCenter and, with the CSR file copied over to vCenter, use the following command:
```
/usr/lib/vmware-vmca/bin/certool --gencertfromcsr --csrfile <CSR_FILE> --cert <CERT_OUT_FILE>
```
7. Copy the contents of <CERT_OUT_FILE>.
8. In the VC UI again, click "Actions" and then "Replace Certificate".
9. Either paste the certificate contents, or upload the <CERT_OUT_FILE> and click "Replace".
There should be an indication of success in the UI via a banner that says, "Certificate was successfully replaced."
Feedback
