Vertica database was shutdown using the adminTools. On a server restart the Vertica does not start. We can see from the adminTools.log:
Then 14:15 the node does not start:2020-03-28 14:14:30.262 admintools/2307:0x7f28252d0700 [adminExec.getRestartPolicy] <INFO> found restartpolicy dict
2020-03-28 14:14:30.262 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <INFO> executing start for DB capm (policy: ksafe); host 122.123.123.123 node v_drdata_node0001
2020-03-28 14:14:30.262 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <INFO> spawn: /opt/vertica/bin/vertica ['/opt/vertica/bin/vertica', '--status', '-D', '/data/verticaDB/catalog/capm/v_drdata_node0001_catalog']
2020-03-28 14:14:33.211 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <WARNING> hostdown: 1 after 1 tries, return code 0
2020-03-28 14:14:33.212 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <WARNING> ksafe but DB not up, skipping
2020-03-28 14:14:33.212 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <INFO> should have started a DB, but didn't
2020-03-28 14:14:33.212 admintools/2307:0x7f28252d0700 [commandLineCtrl.commandHost] <INFO> overall status: 0
All supported releases
The cause was that K-Safe was incorrectly set. As seen in the admintools.conf file:
[Database:capm]
restartpolicy = 1
Restart Policy "Always" not taking effect on single-node cluster and this was fixed by creating an SSH key even in single node installations.
https://www.vertica.com/docs/10.1.x/HTML/Content/Authoring/InstallationGuide/TroubleshootingTheInstall/EnableSecureShellSSHLogins.htm
https://www.vertica.com/docs/10.1.x/HTML/Content/Authoring/InstallationGuide/MCClusterInstall/CreateAPrivateKeyFile.htm
Also, if your database stops abruptly when the Tuple Mover process is still running, the DB will not shut down cleanly.
According to the restart policy Vertica documentation, setting the restart policy requires one to use “K-Safe” for a multi-node cluster and to use “Always” for a single node environment.