During a ZDU, zero downtime upgrade, from 12.2 to 12.3 when WPs are brought up during step 2 of the ZDU, the WPs are throwing forced traces and the PWP logs and traces show the following:
U00005145 MQSet ('1') of the data structure differs from the MQSet ('2') of the Automation Engine.
20210625/154939.429 - U00003422 WARNING: Cannot load conversation memory '0043370c' for server routine 'UCMAIN_R'.
20210625/154939.429 - U00009907 Memory dump 'Input memory' (Address='0x7f72a2ee9010', Length='116')
20210625/154939.429 - 00000000 5356435F 464E4441 30310000 00000000 >SVC_FNDA01......<
20210625/154939.429 - 00000010 A7220000 55433450 4F433131 20202020 >§"..UC4POC11 <
20210625/154939.429 - 00000020 20202020 55433420 57503120 20202020 > UC4 WP1 <
20210625/154939.429 - 00000030= 20202020 20202020 20202020 20202020 > <
20210625/154939.429 - 00000060 20202020 00000000 00000000 00000000 > ............<
20210625/154939.429 - 00000070 00000000 >....<
20210625/154939.429 - UCMAIN_R ADMIN WITH MESSAGE SRERROR
and:
U00005145 MQSet ('1') of the data structure differs from the MQSet ('2') of the Automation Engine.
20210622/094500.479 - U00003422 WARNING: Cannot load conversation memory '0042e8f8' for server routine 'UCVAREXR'.
20210622/094500.479 - U00009907 Memory dump 'Input memory' (Address='0x7f72a2ee9010', Length='1278')
20210622/094500.479 - 00000000 45585354 41525432 30310000 00000000 >EXSTART201......<
20210622/094500.479 - 00000010 45313130 30562D53 46545030 31415F53 >E1100V-SFTP01A_S<
20210622/094500.479 - 00000020 46545020 20202020 20202020 20202020 >FTP <
20210622/094500.479 - 00000030 43495420 20202020 20202020 20202020 >CIT <
20210622/094500.479 - 00000040 20202020 20202020 20202020 20202020 > <
20210622/094500.479 - 00000050 46545041 47454E54 20202020 20202020 >FTPAGENT <
20210622/094500.479 - 00000060 20202020 20202020 20202020 20202020 > <
20210622/094500.479 - 00000070 342E302E 312B6275 696C642E 33352020 >4.0.1+build.35 <
20210622/094500.479 - 00000080 20202020 20202020 20202020 20202020 > <
...
The second shows up for multiple agents
Base version of the Automation Engine is 12.2.6 or older
Target version of the Automation Engine is 12.3.x
This is caused by the ZDU from version before 12.2.9 being not optimal
Both conversation memory errors descried are caused by an issue with the MQSet.
The reason that this happens is that there was an undocumented bug that was resolved with other updates to the ZDU on version 12.2.6 that causes messages to be sent to the wrong "base set" of MQ Tables, or MQSet - so every action on the system should go to the 12.2.6 MQSet, but due to this problem, messages are being sent to 12.3.5 MQSet. Because 12.3.5 does not actually take any actions with MQ Messages, the messages are lost and don't go through.
The following in the PWP log and trace files is an indicator of this:
U00005145 MQSet ('1') of the data structure differs from the MQSet ('2') of the Automation Engine.
Finalizing the ZDU should resolve this error, but the better way to do go about this is to update systems that are on or needing to go to 12.2 to 12.2.9 instead of 12.2.6 and then using the ZDU from there to update to 12.3.x.
12.2.9 ZDU has been optimized and many bugs both public and internal have been fixed with this service pack.