Runaway Connector issue for IDM 3.3.2 and 3.3.3

Products

• VMware Aria Suite

Issue/Introduction

This article provides information to resolve the runaway connector issue with IDM.

Symptoms:

• A check of IDM Admin UI “Total users” count on “User Engagement Dashboard” and compare it with “Users” count on the “Users & Groups” tab usually can identity the issue. If the “Total users” > “Users” count, meaning a lot of service accounts exist, then it indicates the “runaway connector” issue.
• To check further, SSH to each IDM appliance node and check whether the /usr/local/horizon/conf/flags/sysconfig.cloneprep and/or /usr/local/horizon/conf/flags/sysconfig.iamaclone exist. These two flag files are supposed to be deleted after clone process finishes successfully. The existence of these flag files is a proof of the problem.

Environment

VMware Identity Manager 3.3.x

Cause

This issue occurs because the sysconfig.cloneprep and sysconfig.iamaclone files are not getting cleaned up correctly after a cloning operation.

Resolution

To resolve this issue:

1. Remove the two flag files if they exist from each appliance node and reboot the machine.
   
   Note: This step is necessary to stop growing number of connectors and corresponding service accounts and OAuth2Clients.
   
   rm -f /usr/local/horizon/conf/flags/sysconfig.cloneprep
   rm -f /usr/local/horizon/conf/flags/sysconfig.iamaclone
   reboot
    
2. Run the following DB select queries to check DB:
   
   select count(*) from saas.Connector;
   select count(*) from saas.Connector where activationStatus='ACTIVATED';
   select count(*) from saas.Connector where activationStatus='NOT_ACTIVATED';
   select count(*) from saas.ActivationToken;
   select count(*) from saas.OAuth2Client where id<=100 or idIdentityProvider is NOT NULL;
   select count(*), userType from saas.Users where idUserStatus=1 group by userType;
   select count(*) from saas.Users where userType='SERVICE' and idUser not in (select idUser from saas.OAuth2Client) and strEmail='[email protected]';
    
3. Run the following DB select queries to clean up:
   
   a. Clean up the ActivationToken table. All tokens here are for connector activation.
   b. Once a connector is activated, the token should be cleaned.
      
      delete from saas.ActivationToken;
      
   c. Clean up unused OAuth2Client. Each activated connector uses a OAuth2Client to authenticate itself to the IDM service. Unused OAuth2Clients should be cleaned.
   
      delete from saas.OAuth2Client where id not in (select id from saas.OAuth2Client where id<=100 or idIdentityProvider is NOT NULL);
      
   e. Clean up unactivated connectors:
   
      delete from saas.Connector where activationStatus='NOT_ACTIVATED';
      
   f. Clean up auto-generated service user accounts. When OAuth2Client is created, an corresponding service account is also created.
   g. Delete such service users along with unused OAuth2Clients.
   
      delete from saas.Users where userType='SERVICE' and idUser not in (select idUser from saas.OAuth2Client) and strEmail='[email protected]';