VMware Identity Manager Connector may fail to communicate due to config-state.json corruption
search cancel

VMware Identity Manager Connector may fail to communicate due to config-state.json corruption

book

Article ID: 322679

calendar_today

Updated On:

Products

VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

  • Corrupted config-state.json file result in Active Directory login failures
  • Sync connector fails with error - "Failed to parse response received from connector".
  • User receive error "400 bad request - invalid username / password" when using Aria Automation API to authenticate as a domain user
  • Connector Failed to load Auth Adapters | Auth Adapters are missing from VMware Identity Manager node as per this KB 322714
  • Aria Suite Lifecyle Manager shows the globalenvironment environment as healthy
  • Connector sync issue reported when checking health in VMware Identity Manager
  • The UI will not load and the URL will be redirected to https://<vIDMFQDN>/hc/error where it may show an error similar to Error: You do not have permission to access this page: /hc/3104/authenticate/ or  Error : See logs for details
  • Log files /opt/vmware/horizon/workspace/logs/connector.log, /opt/vmware/horizon/workspace/logs/horizon.log show the issue as:
    • com.vmware.horizon.connector.utils.ServiceUtils - Failed to check service health: invalid MOL url
    • ERROR (pool-4-thread-1) [;;;] com.vmware.horizon.connector.admin.service.impl.StartupDirectoryConfigService - could not take the backup as config-state.json file size is empty

Environment

  • VMware Identity Manager 3.3.x
  • Aria Suite Lifecyle Manager 8.x
  • Aria Automation 8.x

Cause

The vIDM service performs periodic backups of the connector configuration to config-state.json. In instances of file system instability or disk space exhaustion, the integrity of this file may be compromised during the write process.

Resolution

Broadcom Product team has been notified and is working to address this issue in a timely manner. Subscribe to this article to receive updates when they are available.

Workaround:

Before following the steps below:
  • Take a snapshot of the VMware Identity Manager Appliance(s)
  • If the UI will not load and users are receiving the /hc/error as shown in the screenshot above in the Introduction section, verify the Bind User/Password of the directory are correct.
    • If a service account is being used in the configuration its password may have expired.
    • Browse to https://vIDM_NODE_FQDN/SAAS/login/0
    • Test and Save the directory configuration once the password has been reset.
    • Skip ahead to step 8 below after receiving a successful Test and Save.
  • If the root partition is full then release disk space Identity Manager root partition is full on one or more nodes in the cluster
 
 
Note that if there are multiple directories in vIDM there will be a directory for each <WORKER_ID> in the <TENANT_NAME> directory. In this case it is necessary to check the config-state.json in every <WORKER_ID> directory and restore the latest stable version for any affected directory . 

  1. SSH to VMware Identity Manager Appliance(s) using root credentials. Change the directory to the location of the config-state.json file by running the command:

cd /usr/local/horizon/conf/states/<TENANT_NAME>/<WORKER_ID>
 
For example  cd /usr/local/horizon/conf/states/VSPHERE.LOCAL/3001      

  1. Stop the service before doing anything.

service horizon-workspace stop

  1. Back up the current configuration file by running the command:

mv config-state.json config-state.json.1

  1. Copy application backup of the configuration file by running the command:

cp -p config-state.json.backup_<latest-stable-version> config-state.json

  1. Change the owner of the config-state.json to horizon user by running the command:

chown horizon:www /usr/local/horizon/conf/states/<TENANT_NAME>/<WORKER_ID>/config-state.json

  1. Change the permission of the config-state.json file by running the command:

chmod 640 /usr/local/horizon/conf/states/<TENANT_NAME>/<WORKER_ID>/config-state.json

NOTE: If the config-state.json file does not get updated with directory information even after restoring from the older versions, recreate the affected directory from the UI. Go to Identity & Access Management > YOUR DIRECTORY and delete the directory and re-add it back. 

  1. Start vIDM/Workspace service by running the command:

service horizon-workspace start
  1. Go to Directory Setting and for each tab do the save operation by navigating to Identity & Access Management > YOUR DIRECTORY > Sync Settings

  1. Navigate each of the tabs and click on Save.
  2. If the "Groups" page refuses to save due to an error about the Bind DN: come back to the directory settings, enter the Bind DN password, validate & save.
    Then come back to save the remaining tabs in Sync Settings.










  1. To avoid the issue for re-occurring please apply vIDM root partition is full on one node of the cluster

Additional Information

Impact/Risks:
  • The config-state.json has been corrupted and needs to be restored. If not restored, the connectors are not going to work in the existing condition
  • Also Please see the following KB (396017) for the same overall error but for IDP configuration value is null in the vIDM node’s config-state.json file
Powered by Wolken Software