EDR: Cluster Does Not Start SupervisorD Service on Master

search cancel

book

calendar_today

Carbon Black EDR (formerly Cb Response)

Datagrid starts on minions but SupervisorD service does not start on the master. Datagrid logs on minions may complain about PGSQL not being started on the master.
No error messages are logged on the master node outside of seeing an attempted services start in /var/log/messages.
SupervisorD logs do not show an attempted service start but individual services can be started by calling on them individually using the /usr/share/cb/cbservice utility.
Note: This is not a common issue that VMWare CB support has encountered very often. If the file count described above is not high ( > 1000 files/directories), then typically startup troubleshooting should happen starting with the services that are failing to start on their respective node.

An unexpected large number of files ('unexpected' in the sense that a service crash may have created additional files) are being scanned for permissions on cb-enterprise startup. Until these files are scanned the services will not start on the master node. Upon a full scan and startup, the Mnesia directory is reset but if thousands of files exist this may take more time than expected, causing a timeout of the services and the cluster to fail to start.
In this case the /var/cb/data/rabbitmq/mnesia directory can be the cause of such an issue. To confirm if this issue is related please confirm the amount of files in the /var/cb/data/rabbitmq/mnesia directory via:

ls -l /var/cb/data/rabbitmq/mnesia | wc -l

If the amount of files/directories in the Mnesia directory is higher than ~400 ( > 1000 files/directories) then proceed to the KB below. If the number of files/directories is not high, this KB is most likely not the solution. Please see the "Related Content" below.
Reset the Mnesia directory by backing it up and then removing it as described in this article: EDR: How to reset Mnesia for RabbitMQ .

thumb_up Yes

thumb_down No