Unable to log in to EEM - ISE_BACKENDDOWN backend is down - EEM failover cluster setup
Article ID: 144478
Updated On:
Products
Issue/Introduction
This causes issues with logging in and/or authorizing with the attached/registered applications.
This document will take you through a step by step to alleviate the issue and assumes a cluster of 2 EEM servers only.
Environment
Failover/HA setup - multiple EEM servers
Cause
This is not a comprehensive list, but if you are seeing a message like the following in the itechpoz_trace.log file under CA Directory:
DSA_E2735 Multiwrite-DISP: Unable to synchronize with peer 'itechpoz-Failover-hostname’
Then this requires manual intervention.
Resolution
The first step will be to disconnect the two EEM servers. This will cause an outage, please plan accordingly and always do this in a test environment first.
This is outlined in the EEM documentation as well.
It is required that the EiamAdmin password be identical on all EEM servers to be in the cluster.
You will use the eiam-clustersetup utility to remove the secondary CA EEM Servers from the primary CA EEM Servers.
After you remove the secondary CA EEM Servers, reset the primary CA EEM Server.
On the primary CA EEM Server, navigate to the following location:
EIAM_HOME/bin
Execute the following command:
java -jar eiam-clustersetup.jar
A confirmation message appears.
Type Y and press Enter.
Execute the following command:
remove
The following message appears.
select hostname
Type the number corresponding to the secondary CA EEM Server that you want to remove and press Enter.
A confirmation message appears.
Type Y and press Enter.
Repeat steps 4 through 6 for all the secondary CA EEM Servers.
Execute the following command
list
Only the primary CA EEM Server is displayed. The secondary CA EEM Servers are removed from the primary CA EEM Server.
To reset the primary CA EEM Server, execute the following command:
resetprimary
The following message appears:
Enter DSA Port [default = 509]
If necessary, update the default DSA port number and press Enter.
The following message appears:
Specify high-availability mode
Select a high-availability mode and press Enter.
A confirmation message appears.
Type Y and press Enter.
The primary CA EEM Server is reset.
To close the eiam-clustersetup utility, execute the following command:
exit
The secondary CA EEM Servers are removed from the primary CA EEM Server and the primary CA EEM Server is reset.
===========================
Once the secondary EEM servers have been removed, stop all EEM services for all servers. The services are the itechpoz router and igateway. For Linux you can run the command dxserver stop all. Verify that there are no running services before proceeding.
===========================
On the primary EEM server, navigate to the \CA\Directory\dxserver\data\itechpoz directory
There are two files here
itechpoz.db
itechpoz.tx
Take a copy of the itechpoz.db file and move this copy to the secondary EEM server in a temp folder.
On the EEM secondary, again navigate to the \CA\Directory\dxserver\data\itechpoz directory
Delete the itechpoz.tx file
Rename the itechpoz.db file to itechpoz.db.orig - make sure in Windows that you have show files extensions enabled, otherwise this step; and everything following; will cause the failure to continue.
Move the itechpoz.db file that was copied from the Primary server to this location.
On your secondary EEM server in the \CA\Directory\dxserver\data\itechpoz directory you will now have these two files:
itechpoz.db
itechpoz.db.orig
===========================
Start the EEM primary services. The itechpoz (dxserver) must be started before igateway.
Verify that you are able to log into EEM to the Global application as EiamAdmin.
If you cannot log in, stop here and call support. At this point, a clean install may be required for the EEM primary server.
If you have verified that you can log in to the Global application as the EiamAdmin user on the Primary server, start the services on the secondary server.
Again, on the secondary server verify that you can log in to the secondary EEM server to the Global application as EiamAdmin.
If you cannot log in, again stop here and call support. A clean install of the secondary may be required.
If you have verified that you are able to log in to the Global application as the EiamAdmin user on the secondary server, leave this up and running and log back on to the primary EEM server.
At this point, we will be re-adding the secondary server as a cluster/failover node as outlined in the EEM documentation.
===========================
Open the command prompt from the primary server and navigate to the EiamInstallation\bin location.
Execute the following command:
java -jar eiam-clustersetup.jar
Type Y and press Enter.
Execute the following command:
add
Type the fully-qualified hostname of the secondary server, and press Enter.
The message "Enter DSA Port [default=509]" appears.
Do one of the following steps:
If you want to accept the default port, press Enter.
If you want to enter a different DSA port, type the port number and press Enter.
Type Y and press Enter.
The failover tool starts validating the entered details. After the validation, the secondary server is added to the primary server.
The next step is to synchronize the configuration of the primary server with the secondary servers in a failover setup.
After the synchronization, the configuration of each secondary server in the failover setup is overwritten with the configuration of the primary server.
Log directly on to the secondary server.
Open the command prompt from a secondary server, and navigate to the EiamInstallation\bin location.
Execute the following command:
java -jar eiam-clustersetup.jar -p <fullyqualifiedhostname_of_primaryserver>
The installation paths of CA EEM, CA Directory, and CA iTechnology iGateway and the status of the failover tool on the secondary server details are displayed. The command-line interpreter changes from $ to the hostname of the secondary server.
Type the EiamAdmin password and press Enter.
Type Y and press Enter.
Execute the following command:
sync
Type the number corresponding to the current hostname, and press Enter.
As you are synchronizing the secondary server with the primary server for the first time, type 1 and press Enter.
Type Y and press Enter.
The failover tool starts validating the synchronization process. After the validation, the secondary server is synchronized with the primary server.
If you are not able to execute the sync command, execute the cluster setup command again on the secondary server.
===========================
At this point, you should now be able to log in to both EEM servers as EiamAdmin to the Global application.
If an external directory has been setup on the Primary server, you will need to manually update the secondary server with this external LDAP connection.
The userstore information does not replicate - any changes have to be done manually to all EEM servers in the cluster.
To verify that replication is working as expected, log in to the primary EEM server as EiamAdmin to the Global application.
Select Manage Access Policies
Create a test policy.
Remember where you created this.
Once the test policy is saved, log out.
Log in to the secondary EEM server as EiamAdmin to the Global application.
Select Manage Access Policies
You should see the test policy you created.
From here, delete the test policy and save.
Log in once again to the primary EEM server as EiamAdmin to the Global application.
Select Manage Access Policies
You should no longer see the test policy.
If all of the above is successful as outlined, the replication is fully functional.
Additional Information
Make sure the passwords are identical by using the KB Article - https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=37739
Feedback
