Workload Automation AE Commands fail with the following message soon after the application server restart:
[autosys@waaeserver1 ~]$ autosyslog -e
CAUAJM_E_10436 Security server unreachable or invalid authentication certificate file.
CAUAJM_E_10434 Error initiating security session.
[EE_BADOBJECT Bad Object]
[ISP_ERROR_NOGATEWAY igateway not running]
[Authenticate Error: Authentication Failed]
[Identity Attempted: ]
[CertificateReader::loadPEM - cannot read certificate]
CAUAJM_W_10406 Control Execute Access Denied!
CAUAJM_E_10434 Error initiating security session.
CAUAJM_W_10440 Class: as-control Resource: ACE.EPLOG User: autosys Access: execute
CAUAJM_W_10442 Time: 1455405684 Delegator: None
[autosys@waaeserver1 ~]$ autorep -J ALL
/*CAUAJM_E_10436 Security server unreachable or invalid authentication certificate file.
CAUAJM_E_10434 Error initiating security session.
[EE_BADOBJECT Bad Object]
[ISP_ERROR_NOGATEWAY igateway not running]
[Authenticate Error: Authentication Failed]
[Identity Attempted: ]
[CertificateReader::loadPEM - cannot read certificate]
*/
The Application Server log ($AUTOUSER/out/as_server.$AUTOUSER)repeatedly logs the below message soon after its restart:
[02/14/2016 04:49:06] CAUAJM_I_40275 Log Rollover level set to <MIDNIGHT,SIZE(100),PURGE(7)>.
[02/14/2016 04:49:06] CAUAJM_I_40244 EnableIPCaching value set to <0>.
[02/14/2016 04:49:06] CAUAJM_I_40244 LogMaxEndLines value set to <0>.
[02/14/2016 04:49:06] CAUAJM_I_40211 Using TZ = Asia/Kolkata.
[02/14/2016 04:49:06] CAUAJM_I_10655 System is running in single server mode. Event server: autosysdb.
[02/14/2016 04:49:11] CAUAJM_I_20197 CA WAAE Application Server operational on port 9000.
[02/14/2016 04:49:11] CAUAJM_I_40244 CA EEM unauthenticated user mode value set to <**UNKNOWN**>.
[02/14/2016 04:49:11] CAUAJM_I_20366 CA WAAE Application Server operational on agent listener port 49156.
[02/14/2016 04:49:11] CAUAJM_I_20367 CA WAAE Application Server operational on auxiliary agent listener port 7500.
ERROR: [0x036ccb70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x036ccb70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x036ccb70] ispUtil.HttpPostRequest: ObtainPC Failed
log4cxx: No appender could be found for logger (PozFactory).
log4cxx: Please initialize the log4cxx system properly.
ERROR: [0x036ccb70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x036ccb70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x036ccb70] ispUtil.HttpPostRequest: ObtainPC Failed
....
...
..
ERROR: [0x036ccb70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x036ccb70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x036ccb70] ispUtil.HttpPostRequest: ObtainPC Failed
ERROR: [0x036ccb70] CertificateReader::loadPEM : etpki_pem_file_to_cert failed [ errorcode : -1 ]
[02/14/2016 04:49:12] CAUAJM_E_10436 Security server unreachable or invalid authentication certificate file.
[02/14/2016 04:49:12] CAUAJM_E_10434 Error initiating security session.
[02/14/2016 04:49:12] CAUAJM_E_10437 Detailed Error Information:
[02/14/2016 04:49:12] [EE_BADOBJECT Bad Object]
[02/14/2016 04:49:12] [ISP_ERROR_NOGATEWAY igateway not running]
[02/14/2016 04:49:12] [Authenticate Error: Authentication Failed]
[02/14/2016 04:49:12] [Identity Attempted: ]
[02/14/2016 04:49:12] [CertificateReader::loadPEM - cannot read certificate]
ERROR: [0x03158b70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x03158b70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x03158b70] ispUtil.HttpPostRequest: ObtainPC Failed
....
...
..
Release: All supported and EEM Multi-Write configured environments
Component: ATSEEM
Configure the WAAE security to work with all nodes of the EEM cluster.
[autosys@waaeserver1 ~]$ autosys_secure
CA WAAE Security Utility
Please select from the following options:
[1] Revert to NATIVE instance security.
[2] Manage CA EEM security settings.
[3] Change database password.
[4] Change remote authentication method.
[5] Manage user@host or user@domain users.
[6] Get encrypted password.
[0] Exit CA WAAE Security Utility.
> 2
Manage CA EEM security settings
Please select from the following options:
[1] Manage CA EEM server settings.
[2] Manage cached credentials.
[9] Exit from "Manage CA EEM security settings" menu.
[0] Exit CA WAAE Security Utility.
> 1
Manage CA EEM server settings
Please select from the following options:
[1] Show current CA EEM server settings.
[2] Set CA EEM server location and regenerate certificate.
[3] Set unauthenticated user mode.
[9] Exit from "Manage CA EEM server settings" menu.
[0] Exit CA WAAE Security Utility.
> 1
CAUAJM_I_60228 CA EEM server: eemserver1
CAUAJM_I_60342 Unauthenticated user mode: OFF
Please select from the following options:
[1] Show current CA EEM server settings.
[2] Set CA EEM server location and regenerate certificate.
[3] Set unauthenticated user mode.
[9] Exit from "Manage CA EEM server settings" menu.
[0] Exit CA WAAE Security Utility.
> 2
Input the CA EEM server name(s) (or hit enter to cancel): eemserver1,eemserver2
Input the CA EEM administrator name (or hit enter to cancel): EiamAdmin
Input the CA EEM administrator password:
Confirm the CA EEM administrator password:
CAUAJM_I_60200 CA EEM certificate generated successfully.
CAUAJM_I_60191 The CA EEM server location was changed successfully.
Please select from the following options:
[1] Show current CA EEM server settings.
[2] Set CA EEM server location and regenerate certificate.
[3] Set unauthenticated user mode.
[9] Exit from "Manage CA EEM server settings" menu.
[0] Exit CA WAAE Security Utility.
> 0
A quick test after security changes:
[02/14/2016 06:59:13] CAUAJM_I_20367 CA WAAE Application Server operational on auxiliary agent listener port 7500.
ERROR: [0x07371b70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x07371b70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x07371b70] ispUtil::TcpConnect: Error in delayed connection() 111 - Connection refused
ERROR: [0x07371b70] ispUtil::TcpConnect Failed to connect to host [eemserver1]
ERROR: [0x07371b70] ispUtil.HttpPostRequest: ObtainPC Failed
log4cxx: No appender could be found for logger (PozFactory).
log4cxx: Please initialize the log4cxx system properly.
CAUAJM_I_10485 CA EEM security session initialized with server <eemserver1,eemserver2>.
[02/14/2016 06:59:14] CAUAJM_W_10472 Resource manager encountered the following during initialization: <Library "libDCAMClient" has not been loaded>.
[02/14/2016 06:59:14] CAUAJM_W_10473 Resource manager will ignore non-virtual resource constraints for all jobs.
[02/14/2016 06:59:14] CAUAJM_I_10474 Resource manager initialization complete.
[02/14/2016 06:59:15] CAUAJM_I_30001 CA WAAE Application Server startup complete.