U00011630 Table UC_HTYP was modified inadmissibly - please contact Automic Support -- After updating to AE 12.3

Products

• CA Automic Workload Automation - Automation Engine
• CA Automic One Automation

Issue/Introduction

After an upgrade from V12.2.2 to V12.3.1 AE system  started successfully, but  agents cannot connect to the system. The Agents process stopp at this point.


In the WP Logs :

=====================================================================

20200914/101448.203 - U00011612 Host 'AGENT1' has the following characteristics: 'Linux' / '3.10.0-1062.4.1.el7.x86_64' / 'x86_64/56' License class: 'S' Version: '12.2.2+hf.2.build.1553005786061'

20200914/101448.203 - U00011635 Host 'AGENT1' has a time difference of '0000000000' seconds to the server.

20200914/101448.218 - U00011630 Table UC_HTYP was modified inadmissibly - please contact Automic Support

20200914/101700.721 - U00011604 Server 'UC4_DEV#WP008': Starting host 'AGENT2' via 'MQ1CP001'.

20200914/101700.721 - U00011612 Host 'AGENT2' has the following characteristics: 'Linux' / '3.10.0-1062.el7.x86_64' / 'x86_64/56' License class: 'S' Version: '12.2.2+hf.2.build.1553005786061'

20200914/101700.721 - U00011635 Host 'AGENT2' has a time difference of '0000000000' seconds to the server.

20200914/101700.736 - U00011630 Table UC_HTYP was modified inadmissibly - please contact Automic Support

 

In the Agent Log:

=====================================================================

200914/094110.123 - U00011630 Table UC_HTYP was modified inadmissibly - please contact Automic Support

20200914/094110.123 - U02003073 Agent Prozess 'AGENT,PID=14731' shutdown has been initiated.

20200914/094110.124 - U02002040 Disconnected from '*IPC(LISTENER)' (socket handle = 's=9,ID=4').

20200914/094110.124 - U02000010 Connection to Server '*SERVER(s=10,ID=71)' terminated.

20200914/094110.126 - U02000041 Shutdown Agent 'AGENT2'.

 

In the Service Manager log:

=====================================================================

0200120/130721.596 - U00022004 Starting process 'UC4_DEV AGENT2 Linux' ...

20200120/130721.596 - U00022005 ... with command line  '/var/scheduling/dev/bin/ucxjlx6 /var/scheduling/dev/AGENT2-dev.ini'.

20200120/130721.596 - U00022006 ... and start directory  '/var/scheduling/temp'.

20200120/130721.608 - U00022007 Process 'UC4_DEV AGENT2 Linux' (ID '14731') successfully started.

20200914/094110.134 - U00022012 Process 'UC4_DEV AGENT2 Linux' (ID '14731') ended.

20200914/094110.134 - U00022022 Process 'UC4_DEV AGENT2 Linux' ended, exit code='0'.

 

The SQL-Statement "Select count(*) from UC_HTYP' returns a values that is greater than 1000. In 12.3 the same statement must return 43.

 

=====================================================================

Background:

In version 12.3 Telemetry was introduced and the use of licenses became obsolete. In the former version the HTYP data were necessary to validate the use particular license classes with specific system components. In Version 12.3 that the use of the data in the UC_HTYP table  is not longer the same, but it must be consistent, otherwise agents are not able to connect to the AE system.

During the loading of the 12.3 intial data, the DBLoad Utility deletes the content of the table UC_HTYP and inserts the mandatory 43 required standard datasets.

 

Environment

Release : 12.3 Component : AUTOMATION ENGINE

Cause

Something went wrong during the database upgrade. The UC_HTYP table was not emptied before loading the 12.3 specific datasets. As most of the time the tracing mode database=4 was not activated it is difficult to find the root cause.

 

 

Resolution

1. Run the load of the Initial Data again with DBLoad, this the trace flag database=4.

If this time the deleted of the obsolete dataset is successful following messages should be present in the traces:

 

==========================================================

20200914/120212.336 - delete from UC_HTYP 
20200914/120212.460 - UCUDB32 EXEC RET 0000 HSTMT: 0x007f5f09a4de10 VALUE: 0x00000000000565 ALL:  0.12331 DB:  0.12326 ODBC:  0.00001 UDB:  0.0000

 

2. Restart all agents manually.

 

Another workaround consists in truncating the UC_HTYP table and importing the UC_HTYP table from a 12.3 system that does not have the problem. But this kind of actions should ONLY be done only when validated by the Broadcom Support Team.

 

IMPORTANT: This problem is initially a bug detected in AWA 12.2.2. The bug is corrected in version 12.2.6. In the database in versions < 12.2.6 the cleaning up of the UC_HTYP was not correctly performed. Most of the time the problem is detected when DBLoad 12.3.x runs the first database update. Running the import of initial data will work because database is then in version 12.3.3, a version where the bug is corrected and UC_HTYP will be updated as required.