Greenfield VCF 9.1 deployment fails on NSX side "UNABLE_TO_ISSUE_VMCA_SIGNED_CERTIFICATE"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

In a Greenfield VCF 9.1 deployment, the install process fails on NSX side, with the following error message "UNABLE_TO_ISSUE_VMCA_SIGNED_CERTIFICATE"
This scenario can happen on all, or just some of the deployed NSX nodes. 

less -i domainmanager.log

<Date && Time> DEBUG [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.n.s.c.c.ApiConnection,dm-exec-241]  Created ApiClient connection to: <impacted Node>
<Date && Time> DEBUG [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.n.s.c.c.ApiConnection,dm-exec-241]  NSX Version: 9.1.0.0100.25470810, NSX Version with policy baseline: 4.1.0.0.0-0
<Date && Time> DEBUG [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.n.s.c.c.ApiConnection,dm-exec-241]  Overriding Mp handlers with policy handlers
<Date && Time> DEBUG [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.n.s.c.c.NsxtManagerCertOperations,dm-exec-241]  Starting applying API certificate with - ID: 49386873-3c24-440a-b77b-25aa1725361c; nodeId: 86df1242-6c1d-8917-21dc-4f79e99c611e
<Date && Time> DEBUG [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.n.s.c.c.ApiConnection,dm-exec-241]  Closed ApiClient connection.
<Date && Time> ERROR [vcf_dm,6a3a8fa38a156c8bd0653afa4974e641,45aa] [c.v.v.c.f.p.n.a.IssueVMCACertsForNsxtManagerAction,dm-exec-241]  UNABLE_TO_ISSUE_VMCA_SIGNED_CERTIFICATE
java.lang.RuntimeException: Retriable operation 'Issuing certificate to NSX manager: <impacted Node>' failed to complete after 3 retries.
        at com.vmware.evo.sddc.common.util.RetriableCallable.call(RetriableCallable.java:221)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.IssueVMCACertsForNsxtManagerAction.issueCertificate(IssueVMCACertsForNsxtManagerAction.java:145)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.IssueVMCACertsForNsxtManagerAction.lambda$execute$0(IssueVMCACertsForNsxtManagerAction.java:100)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.IssueVMCACertsForNsxtManagerAction.execute(IssueVMCACertsForNsxtManagerAction.java:98)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.IssueVMCACertsForNsxtManagerAction.execute(IssueVMCACertsForNsxtManagerAction.java:37)
[...]
        at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.vmware.vapi.std.errors.InvalidRequest: InvalidRequest (com.vmware.vapi.std.errors.invalid_request) (statusCode:400) => {
    messages = [],
    data =  => {error_message=There are pending certificate batch operations., httpStatus=BAD_REQUEST, error_code=2156, module_name=internal-framework},
    errorType = INVALID_REQUEST
}
        at com.vmware.vapi.std.errors.InvalidRequest._newInstance2(InvalidRequest.java:195)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at com.vmware.vapi.internal.bindings.convert.impl.JavaClassStructConverter.createStructBinding(JavaClassStructConverter.java:282)

Environment

VCF 9.1 

Resolution

1. Check/ amend the resolv.conf file, on all 3 NSX Nodes to match the SDDC installer one, including the search for the domain
   

   cat /etc/resolv.conf
   
   nameserver <DNS_server_1>
   nameserver <DNS_server_2>
   nameserver 127.0.0.53
   search <domain>.<TBD>

2. restart task from UI