Failed CA IDM Provisioning Tasks: Failed to activate connector type on proxy connector server, Failed to connect to proxy connector server

Products

CA Identity Manager CA Identity Governance CA Identity Portal CA Identity Suite

Issue/Introduction

Many tasks have failed in CA IDM.

Looks like most of them were trying to provision or de-provisioning AD groups from a specfic domain.
We have already synced all of the failed tasks and trying to investing the root cause of this.
Mentioned below are the log information of Connector Server:

2020-01-31 06:00:18,274 11450868 [ApacheDS Worker-thread-74] (com.ca.jcs.apacheds:org.apache.directory.server.ldap.support.SearchHandler:505) DEBUG - *** id=38/10.61.150.70:32750: Search: END: ERROR: org.apache.directory.shared.ldap.exception.LdapNameNotFoundException: [email protected]: The endpoint type eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa that was referenced in the request has not been activated
2020-01-31 06:00:18,290 11450884 [ApacheDS Worker-thread-76] (com.ca.jcs.core:com.ca.jcs.osgi.exchange.router.MessageRouter:551) ERROR - A remote server returned an error: org.apache.directory.shared.ldap.exception.LdapNameNotFoundException: [email protected]: The endpoint type eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa that was referenced in the request has not been activated
:1226) DEBUG - Transformed message : LdapMessage
    message Id : 40
    Add Response
        Ldap Result
            Result code : (UNWILLING_TO_PERFORM) unwillingToPerform
            Matched DN : ''
            Error message : 'code 53 (UNWILLING_TO_PERFORM): failed to add entry eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa: [email protected]: Failed to activate connector type on proxy connector server: [email protected]: Failed to connect to proxy connector server: ldap://localhost:20402'

2020-01-31 06:00:18,368 11450962 [ApacheDS Worker-thread-78] (com.ca.jcs.apacheds:org.apache.directory.shared.ldap.codec.TwixEncoder:117) DEBUG - Encoding this LdapMessage : LdapMessage
    message Id : 40
    Add Response
        Ldap Result
            Result code : (UNWILLING_TO_PERFORM) unwillingToPerform
            Matched DN : ''
            Error message : 'code 53 (UNWILLING_TO_PERFORM): failed to add entry eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa: [email protected]: Failed to activate connector type on proxy connector server: [email protected]: Failed to connect to proxy connector server: ldap://localhost:20402'
2020-01-31 06:00:18,482 11451076 [ApacheDS Worker-thread-80] (com.ca.jcs.core:com.ca.jcs.DefaultConnectorManager:644) ERROR - The endpoint type eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa that was referenced in the request has not been activated

2020-01-31 06:00:18,576 11451170 [ApacheDS Worker-thread-84] (com.ca.jcs.csproxy:com.ca.jcs.csproxy.CSProxyMetaConnectorType:378) ERROR - Failed to activate connector type on proxy connector server

Resolution

The etatrans.log indicates that the failures were caused by the java connector server not being able to connect to the C++ connector server. The C++ server, per the logging below, refers to this connector server:

ldap://localhost:2040.

20200131:060028:TID=4f0b40:Add :S922:C916:S: Connector Server Add (eTNamespaceName=ActiveDirectory) Requested by User 007007 -
20200131:060028:TID=4f0b40:Add :S922:C916:S:+ TenantNotSet
20200131:060028:TID=4f0b40:Add :S922:C916:P: URL: ldaps://idm14-abc.net:20411
20200131:060028:TID=4f0b40:Add :S922:C916:P: dn: eTNamespaceName=ActiveDirectory,dc=IM
20200131:060028:TID=4f0b40:Add :S922:C916:P: eTNamespaceName: ActiveDirectory
20200131:060028:TID=4f0b40:Add :S922:C916:P: eTAgentPluginDLL: W2KNamespace.dll
20200131:060028:TID=4f0b40:Add :S922:C916:P: objectClass: eTADSNamespace
20200131:060028:TID=4f0b40:Add :S922:C916:F: FAILURE: Connector Server Add (eTNamespaceName=ActiveDirectory)
20200131:060028:TID=4f0b40:Add :S922:C916:F: rc: 0x0035 (DSA is unwilling to perform)
20200131:060028:TID=4f0b40:Add :S922:C916:F: msg: Connector Server Add failed: code 53 (UNWILLING_TO_PERFORM): failed to a
20200131:060028:TID=4f0b40:Add :S922:C916:F:+dd entry eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa: [email protected]: Faile
20200131:060028:TID=4f0b40:Add :S922:C916:F:+d to activate connector type on proxy connector server: [email protected]: Failed
20200131:060028:TID=4f0b40:Add :S922:C916:F:+ to connect to proxy connector server: ldap://localhost:20402(ldaps://idm14-abc.ad.net:20411):0200131:060028:TID=4f0b40:Add :S922:C916:F:+(ldaps://idm14-abc.ad.net:20411)
20200131:060028:TID=4f0b40:Modify :C916:C752:I: COMPLETED JOB 1: AddJob [eTNamespaceName=ActiveDirectory, rc=53(Connector Serve
20200131:060028:TID=4f0b40:Modify :C916:C752:I:+r Add failed: code 53 (UNWILLING_TO_PERFORM): failed to add entry eTNamespaceName
20200131:060028:TID=4f0b40:Modify :C916:C752:I:+=ActiveDirectory,dc=IM,dc=etasa: [email protected]: Failed to activate connector
20200131:060028:TID=4f0b40:Modify :C916:C752:I:+type on proxy connector server: [email protected] <--This JCS server is the one making the request <: Failed to connect to proxy con

20200131:060028:TID=4f0b40:Modify :C916:C752:I:+nector server: ldap://localhost:2040 <--refers to the C++ server (ldaps://idm14-abc.ad.net:20411)
20200131:060028:TID=4f0b40:Modify :C916:C752:I:+)], status=ERROR
20200131:060028:TID=4f0b40:Modify :C916:C752:I: Connector Initialization FAILED
20200131:060028:TID=4f0b40:Modify :C916:C752:F: FAILURE: Child Modify (eTADSAccountName=Olivia, Olive)
20200131:060028:TID=4f0b40:Modify :C916:C752:F: rc: 0x0035 (DSA is unwilling to perform)
20200131:060028:TID=4f0b40:Modify :C916:C752:F: msg: :ETA_E_0008<MAC>, Active Dir. Account 'Olivia, Olive' on 'AD-SB' modif
20200131:060028:TID=4f0b40:Modify :C916:C752:F:+ication failed: Connector Server Add failed: code 53 (UNWILLING_TO_PERFORM): fail
20200131:060028:TID=4f0b40:Modify :C916:C752:F:+ed to add entry eTNamespaceName=ActiveDirectory,dc=IM,dc=etasa: [email protected]
20200131:060028:TID=4f0b40:Modify :C916:C752:F:+: Failed to activate connector type on proxy connector server: [email protected]:
20200131:060028:TID=4f0b40:Modify :C916:C752:F:+ Failed to connect to proxy connector server: ldap://localhost:20402 (ldaps://idm...

The C++ service may not have been running or was experiencing problems. Did you need to restart any services to get processing to continue? You can check the im_ps.log to see if it logged any errors. However, if the requests from JCS never made it to the C++ server there will be nothing in the logs.