Errors reading catalog when starting Data Repository Vertica database in DX NetOps Performance Management

Products

• CA Performance Management - Usage and Administration
• DX NetOps

Issue/Introduction

A hardening script was run against the DX NetOps Performance Management Data Repository Vertica database.

When testing stop and start cycles for the database it stops fine, but when starting it the following errors are seen in the admintools.log file.

2021-04-29 10:59:19.339 at_exec/19016:0x7f1b695bf7c0 [CatalogEditor.sendCmd] <ERROR> Exception encountered while running catalog editor
Traceback (most recent call last):
  File "/opt/vertica/oss/python/lib/python2.7/site-packages/vertica/tools/CatalogEditor.py", line 120, in sendCmd
    self.ceproc.stdin.write(cmd_string)
IOError: [Errno 32] Broken pipe
2021-04-29 10:59:19.339 at_exec/19016:0x7f1b695bf7c0 [CatalogEditor.sendCmd] <ERROR>
Vertica Catalog Editor: broken pipe detected
Added stdout messages to error.
BEGIN CE output
END CE output
2021-04-29 10:59:19.340 at_exec/19016:0x7f1b695bf7c0 [ATRunner.exec_module] <ERROR> command got exception: Could not load from Catalog Editor.
 Catalog Editor state
CatalogEditor instance
Closed? False
CE subprocess = <subprocess.Popen object at 0x7f1b62062d90>
CE proc.poll = -6 CE proc.stdin = <open file '<fdopen>', mode 'wb' at 0x7f1b62032b70> CE proc.stdout = <open file '<fdopen>', mode 'rb' at 0x7f1b62032ae0> Exception was Broken pipe
Vertica Catalog Editor: broken pipe detected
Added stdout messages to error.
BEGIN CE output
END CE output
2021-04-29 10:59:19.358 admintools/18981:0x7f552b3a87c0 [compute_vdatabase.handle_results] <INFO> Handling results for compute_vdatabase
2021-04-29 10:59:19.359 admintools/18981:0x7f552b3a87c0 [NewSSH.computeVDatabase] <WARNING> There was a problem running the catalog editor on host 10.x.x.x. If all hosts have problems, then the database will not be able to start. Problem details: <ATResult> status=1 host=10.44.205.17 content={}
2021-04-29 10:59:19.359 admintools/18981:0x7f552b3a87c0 [adminExec.display] <INFO> Unable to read database catalogs - cannot start database.
2021-04-29 10:59:19.359 admintools/18981:0x7f552b3a87c0 [commandLineCtrl._logPrint] <INFO> Database drdata did not start successfully: Unable to read database catalogs

Environment

All supported DX NetOps Performance Management Data Repository environments

Cause

The hardening script enabled FIPS security at the OS level. This is preventing the Vertica services from properly accessing and reading the catalog data necessary for start up and normal operation of the database.

Resolution

Disable FIPS at the OS level.