Directory sync in vRA 7.x takes very long or fails but vRA is scaled according Requirements

Products

• VMware
• VMware Aria Suite

Issue/Introduction

Symptoms:

• In an IWA directory Configuration, running the Directory sync in vRealize Automation 7.x fails.
• In the /storage/log/vmware/horizon/connector.log file, you see the connection reset messages similar to:

2017-11-16 10:17:37,516 WARN  (SimpleAsyncTaskExecutor-240) [[email protected];[email protected];127.0.0.1] com.vmware.horizon.directory.ldap.LdapConnector - Failed to connect to DC1.domain.domroot.lan:389
javax.naming.CommunicationException: SASL bind failed: DC1.domain.domroot.lan:389 [Root exception is java.net.SocketException: Connection reset]
    at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:242)
    at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2788)
    at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:319)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:192)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:210)
    at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:153)
    at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:83)
    ...
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at com.sun.jndi.ldap.Connection.run(Connection.java:860)

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vRealize Automation 7.1.x
VMware Identity Manager 2.x
VMware vRealize Automation 7.3.x
VMware vRealize Automation 7.2.x
VMware vRealize Automation 7.0.x

Cause

The actual cause can vary from Product & Network configuration and further troubleshooting is required.

Resolution

To resolve this issue:

• If UDP has been blocked in the network, then add udp_preference_limit = 1 in the file /etc/krb5.conf.

[libdefaults]
  udp_preference_limit = 1
  ...

 

• If the connection issues have been identified due to slow response from AD Server or AD Server from Remote Location, see VMware Identity Manager is slow during Active Directory sync (2144953). 
• If the connection is limited to certain Domain Controller,  then modify the /etc/krb5.conf to contain the relevant Domain Controller:

[libdefaults]
  ...
  dns_lookup_kdc = false
  #pkinit_kdc_hostname = <DNS>
  ...
  
[realms]
  DOMAIN1.LOCAL = {
   auth_to_local = RULE:[1:$0\$1](^domain1\.LOCAL\\.*)s/^domain1\.LOCAL/domain1/
   auth_to_local = RULE:[1:$0\$1](^domain1\.LOCAL\\.*)s/^domain1\.LOCAL/domain1/
   auth_to_local = RULE:[1:$0\$1](^domain3\.NET\\.*)s/^domain3\.NET/domain3/
   auth_to_local = RULE:[1:$0\$1](^domain2\.DOMROOT\.INTERNAL\\.*)s/^domain2\.DOMROOT\.INTERNAL/domain2/
   auth_to_local = DEFAULT
   kdc = kdc1.domain1.local
   kdc = kdc2.domain1.local
  }
  DOMAIN2.DOMROOT.INTERNAL = {
   kdc = kdc1.domroot.internal
   kdc = kdc2.domroot.internal
  }    
  DOMAIN3.NET = {
   kdc = kdc1.domain3.net
   kdc = kdc2.domain3.net
  }

• ​Ensure that /etc/krb5.conf file has ​Permissions set to 644 & Ownership root:root.

Additional Information

Impact/Risks:
It is recommended to either create a new tenant for troubleshooting or possible deploy a single vRA Node for Troubleshooting