Install error during RU MDB Patch update

Products

CA Service Desk Manager CA Service Management - Service Desk Manager

Issue/Introduction

Following the install of Service Management 17.3 RU4 or RU5, errors may present in the stdlogs that reference table usp_mailbox_oauth:

01/01 00:00:00.00 SERVER sqlagt:select18 8496 ERROR orclclass.c 1313 SQL Statement Failed. ORA: 942 Message: ORA-00942: table or view does not exist

01/01 00:00:00.00 SERVER sqlagt:select18 8496 ERROR orclclass.c 1314 SELECT count(*) FROM usp_mailbox_oauth WHERE usp_mailbox_oauth.mailbox = XXXX

Similar errors may also appear in the stdlogs as seen here from a SDM 17.3 RU4 installation:

bpvirtdb_srvr 6880 ERROR vdbmisc.c 688 Miscellaneous Database error occured :[Microsoft SQL Server Native Client 11.0] [ SQL Code=208 SQL State=42S02] Invalid object name 'usp_mailbox_oauth'.

web:wsp 5320 ERROR sel_data_cache.c 1116 Error in mailbox_oauth Select_Cache method got_initial_count: AHD04199:An unexpected Database error occurred. Contact your administrator.

If one runs pdm_extract on the given table, ie:

pdm_extract usp_mailbox_oauth

There is a DBCallback error message presented. The backend MDB database shows that the usp_mailbox_oauth table is missing as well.

Environment

Release : 17.3

Component : SVC DESK UPGRADE

Cause

The issue is being caused by a previous 17.3 RU3 patch instance. During the installation of 17.3 RU3, the installer will create a "cum3_3" directory in the SDM install folder's patches directory and will place the MDB patch update, either ORACLE_MDB.CAZ or MSSQL_MDB.CAZ in this directory, where the CAZ file will be uncompressed to create the DB patch update in the "cum3_3" directory. What happens is that the RU4 and RU5 patch updates will write its own copy of ORACLE_MDB.CAZ or MSSQL_MDB.CAZ to the same "cum3_3" directory, but as there will be components from the previous 17.3 RU3 MDB update in this directory, the attempt to uncompress the ORACLE_MDB.CAZ or MSSQL_MDB.CAZ file that the 17.3 RU4/RU5 will fail.

In the install.log, under %TEMP%/casm directory, one will find the indications of the 17.3 RU3 patch install writing to the cum3_3 directory:

2021/01/01 00.00.00.568 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.3 ] [CommonProcessUtilities] Computer Associates CAZIPXP Version 1.8

2021/01/01 00.00.00.569 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.3 ] [CommonProcessUtilities]

2021/01/01 00.00.00.569 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.3 ] [CommonProcessUtilities] Uncompressing C:\Program Files (x86)\CA\Service Desk Manager\patches\cum3_3\MSSQL_MDB.CAZ ...

2021/01/01 00.00.00.569 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.3 ] [CommonProcessUtilities] findcommon.bat

2021/01/01 00.00.00.573 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.3 ] [CommonProcessUtilities] included_manifest

However, the same install.log will show the same cum3_3 directory being used to place the 17.3 RU5 MDB update, but it will fail to decompress:

2021/01/01 00.00.00.684 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [CommonProcessUtilities] Computer Associates CAZIPXP Version 1.8

2021/01/01 00.00.00.684 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [CommonProcessUtilities] Uncompressing C:\Program Files (x86)\CA\Service Desk Manager\patches\cum3_3\MSSQL_MDB.CAZ ...

2021/01/01 00.00.00.684 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [CommonProcessUtilities] findcommon.bat

2021/01/01 00.00.00.690 DEBUG [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [CommonProcessUtilities] Error creating or opening C:\Program Files (x86)\CA\Service Desk Manager\patches\cum3_3/findcommon.bat [5:Access is denied.]

2021/01/01 00.00.00.692 INFO [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [ActiveProcessHolder] Exit Monitor. Process ID (PID): 5584

2021/01/01 00.00.00.692 INFO [DeployThread: Applying CA Service Desk Manager MDB Update(s) : 17.3.0.5 ] [CommonProcessUtilities] Command terminated with error

Resolution

Dev/SE has been made aware of this issue in a defect that indicates the re-usage of the "cum3_3" directory will affect the patch installation for environments that had 17.3 RU3 in place initially and are applying 17.3 RU4 or RU5.

To address the issue in an existing environment in which 17.3 RU3 was previously applied and 17.3 RU5 is already present, one will need to re-do the execution of the 17.3 RU5 installation process. It should not have any effect on the existing files that were placed in the earlier 17.3 RU5 install attempt and will require additional steps before execution in order to prepare the environment for the reinstall of 17.3 RU5

Note: As the intent of these instructions are to re-do the MDB update provided in 17.3 RU5, these instructions should be executed just one time on either the BG or Primary SDM Server. The constituent servers can be left out, but all SDM Services should be shut down before the installation work is done, and all appropriate backups taken beforehand.

- Confirm that the given server shows the same problem with accessing table usp_mailbox_oauth. Run this command on while SDM Services are running on the local Admin command prompt:

pdm_extract usp_mailbox_oauth

A successful command will return a row count of 0 or more. Unsuccessful will show a "dbcallback" error message.

- Back up the entire SDM Server and database (snapshots if these are VM's)

- Stop all SDM Services.

- On the SDM server, locate the SDM install's "patches" directory, ie: E:\CA\SERVIC~1\patches

- Back up the patches folder to another location on the server, then empty out the E:\CA\SERVIC~1\patches directory entirely (this includes the E:\CA\SERVIC~1\patches\cum3_3 directory)

E:\CA\SERVIC~1\patches should be present, but the directory should be empty

- Locate the %TEMP% folder, which should be C:\Users\ADMINI~1\AppData\Local\Temp

- Back up the %TEMP%\casm folder to another location on the server, then remove the C:\Users\ADMINI~1\AppData\Local\Temp\casm directory entirely.

C:\Users\ADMINI~1\AppData\Local\Temp\casm directory should be gone entirely.

- Go into the MDB database and run/commit this DB query:

UPDATE al_cdb_componentinstallstate SET installationstate = 'failed' where packageid LIKE '%17.3.0.5%'

This will set all entries for the 17.3 RU5 install to a 'failed' state. This is important as we are essentially re-running the 17.3 RU5 patch install.

- Reboot the given SDM Server.

- Confirm that the SDM services are completely down (pdm_status should indicate "The daemons are not running")

- Run the 17.3 RU5 update as before.

- After the 17.3 RU5 update completes, start SDM Services up, then run this command on a local admin command prompt to confirm that the table is present:

pdm_extract usp_mailbox_oauth

Additional Information

The given table, usp_mailbox_oauth, was introduced in 17.3 RU4 and is carried over in 17.3 RU5.