CA Service Desk Manager (ITSM) "AHD50050:DB confirmation failed. Reason: AHD50060:Database check FAILED."
Article ID: 135986
Updated On:
Products
Issue/Introduction
This is a general database error, indicating a severe connection issue to the ITSM database:
"AHD50050:DB confirmation failed. Reason: AHD50060:Database check FAILED."
It typically is the end symptom seen, rather than being the root cause.
There are typically other error messages around database issues in this or related logs. It is these related symptoms which must be investigated to understand root cause.
WARNING:
This can be a severe issue, requiring immediate attention. Total loss of CA ITSM application operation is likely.
Environment
Release : 17.1
Component : SERVICE DESK MANAGER
Cause
Resolution
Investigation of the root cause is required.
Please stop the CA ITSM system for troubleshooting. Engage the CA SDM Application Administrator and if possible, the CA SDM Database Administrator.
TROUBLESHOOTING STEPS
1) General symptoms seen. For example, CA SDM Windows Services do not start at all. Or else they do start, but there are CA SDM web client error messages thrown during use. Collect general troubleshooting information such as the time the issue was first noted.
2) Review the CA SDM /log folder for all files updated at the time of the above message.
Please note that the error message is generic and indicates a database connection issue. It it important to look into the surrounding logs and environment for any other information that may point to the underling root cause.
EXAMPLE
This example is from a client site, where the first issue noted was that the CA SDM "Windows Service" was not present.
Further investigation revealed that the pdm_configure process would not run, as it could not connect to the database. It is the pdm_configure process which adds in the CA SDM Windows Service if needed.
This led to a log review and a general amalgamation of errors which pointed to this root cause:
- Key tables were missing, had columns missing, and had data missing.
The root cause in other cases with this generic message may well be different.
Here are the log files to illustrate this problem, and how they intersect with each other.
AT STARTUP
----------------------------
The generic message is noted, after an attempt to connect to the database.
08/16 00:37:12.60 MY_SERVER proctor_MY_SERVER 4496 SIGNIFICANT pdm_process.c 575 Process Started (4832):D:/PROGRA~2/CA/SERVIC~1/bin/confirm_db
08/16 00:37:12.67 MY_SERVER sqlagt:Default 4364 SIGNIFICANT sql_agent.c 245 STARTUP of sql_agent:mdb:key_prov_nxd:Default
08/16 00:37:12.82 MY_SERVER proctor_MY_SERVER 4496 SIGNIFICANT pdm_process.c 927 Process stopped (D:/PROGRA~2/CA/SERVIC~1/bin/ddict_rd -t return: 0
08/16 00:37:12.82 MY_SERVER proctor_MY_SERVER 4496 SIGNIFICANT pdm_process.c 936 Stopped: D:/PROGRA~2/CA/SERVIC~1/bin/ddict_rd -t
08/16 00:37:13.53 MY_SERVER confirm_db 4832 SIGNIFICANT ntservice.c 311 Command return non-zero exit code. Command: redirect -m "C:\Windows\TEMP\tmp38" "C:\Windows\TEMP\tmp39" "D:/PROGRA~2/CA/SERVIC~1/bin/sql_check_db" -c -s MY_SERVERESK >"C:\Windows\TEMP\tmp38" 2> "C:\Windows\TEMP\tmp39" . Exit Code: 99
08/16 00:37:13.60 MY_SERVER pdm_proctor_nxd 2288 SIGNIFICANT STDIO 0 AHD50050:DB confirmation failed. Reason: AHD50060:Database check FAILED.
08/16 00:37:13.60 MY_SERVER pdm_proctor_nxd 2288 SIGNIFICANT STDIO 0 AHD50051:Server shutdown requested.
08/16 00:37:13.62 MY_SERVER confirm_db 4832 SIGNIFICANT api_misc.c 585 Requesting shutdown of system. Reason (AHD50060:Database check FAILED. )
08/16 00:37:13.62 MY_SERVER slump_nxd 4064 EXIT server.c 2857 Slump_nxd was requested to shutdown. Check Prior log information.
CONFIGURE.0
--------------------------
Issues with SQL tables and columns was found after a pdm_configure was attempted. Direct examination of these tables afterwards in the database and comparison against the DocOps table definitions confirmed that tables, columns and data were missing.
08/16 14:53:49.48 MY_SERVER dbload 1408 SIGNIFICANT dbload.c 647 Dbload Started - Parms: -M "D:\PROGRA~2\CA\SERVIC~1\data\docres.dat"
08/16 14:53:49.62 MY_SERVER sql_agent 4828 SIGNIFICANT sql_agent.c 245 STARTUP of sql_agent:mdb:dbload-#1408:
08/16 14:53:50.04 MY_SERVER sqlagt:Default 5040 ERROR sqlsrvr.c 2704 Procedure (SDM_KEY_MGR) was not found
08/16 14:53:50.04 MY_SERVER key_prov_nxd 3656 ERROR key_control.c 960 Couldn't execute stored procedure: 15
08/16 14:53:50.29 MY_SERVER key_prov_nxd 3656 SEVERE_ERROR value.c 424 Bad value type1598
08/16 14:53:50.29 MY_SERVER key_prov_nxd 3656 SEVERE_ERROR value.c 424 Bad value type1701636
08/16 14:53:50.30 MY_SERVER sqlagt:Default 5040 ERROR sqlsrvr.c 2704 Procedure (SDM_KEY_SEED) was not found
08/16 14:53:50.30 MY_SERVER key_prov_nxd 3656 ERROR key_control.c 960 Couldn't execute stored procedure: 15
08/16 14:53:50.33 MY_SERVER sql_agent 4828 ERROR sqlclass.c 1010 SQL Execute failed: [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'servlet_server'.; [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'rep_server'.; [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'aa_services'.
08/16 14:53:50.33 MY_SERVER dbload 1408 ERROR dbintf.c 757 Db callback event:4 error:15
08/16 14:53:50.33 MY_SERVER dbload 1408 ERROR dbintf.c 761 SELECT #aa_services , #archive_path , #archive_type , #cgi_path , #default_rep , #del , #description , #file_limit_size , #prohibited_ext , #protocol , #rep_server , #repository_type , #retrieve_path , #servlet_server , #sym , #upload_path , #persid , #id FROM Document_Repository
08/16 14:53:50.34 MY_SERVER sql_agent 4828 ERROR sqlclass.c 1010 SQL Execute failed: [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'servlet_server'.; [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'rep_server'.; [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'aa_services'.
CONFIGURE.0 log
----------------------------
The configure log provided similar information (this is a different time stamp, but similar message):
09/17 05:28:15.810 INFO ? ? ERROR: AHD50072: Unable to log in to SQL Server
09/17 05:28:16.121 INFO ? ? Process sql_check_db completed with exit code 99
09/17 05:28:16.121 INFO SQLDatabasePage.java 657 Running: sqlcmd -E -S SDMTEST -l 5 -i tmp_sql.sql
09/17 05:28:16.508 INFO SQLDatabasePage.java 678 Changed database context to 'master'.
CHECKDB.0
----------------------------
The database check also failed.
08/16 00:37:13.43 MY_SERVER sql_ckeckdb 4556 ERROR sqlclass.c 1010 SQL Execute failed: [Microsoft SQL Server Native Client 11.0] [ SQL Code=207 SQL State=42S22] Invalid column name 'server_type'.
08/16 00:37:13.45 MY_SERVER sql_ckeckdb 4556 ERROR sqlclass.c 1011 select local_host from usp_servers where server_type = 0
RESOLUTION
The exact resolution will depend on the fault that exists and the data available to effect a repair.
1) This scenario required restoration of the missing data. This may be possible to get from a network backup of the database, a database backup, a test system which has similar data, or a newly built test system with similar data.
2) Then the pdm_configure can be run again, and the CA ITSM Windows Services restarted.
TIP: The error logs will tell you the tables in question, but some common tables which can give rise to this error are usp_servers and dlgtsrv because they contain the server information, which lets CA ITSM connect to the database. If these are wrong or missing, then this error is likely to result.
Additional Information
The ITSM 17.2 table definitions can be found here:
Feedback
