S500 does not come back up after a reboot

Products

• Security Analytics

Issue/Introduction

After a reboot, the Security Analytics S500 appliance does not come back up.  It may take a manual reboot for the server to come back up.

Cause

It is possible that serial 'garbage' is interfering with the serial connection on the appliance.  This can be caused by an attached serial device sending unsolicited characters to the Security Analytics appliance.

Resolution

1. Connect the appliance's serial console port to a continuously-available terminal (preferably with logging).  This precludes unsolicited or unknown serial garbage on the line. It also gives you a chance (via logging) to capture any console messages that might be emitted when the appliance operating system hangs.
2. Disconnect entirely the appliance's serial console port from a presumed serial concentrator of some kind.  This is helpful for troubleshooting purposes to see if removing all devices solves the problem.
3. Ensure that any serial device attached to the appliances console port will never send unsolicited characters to the appliance.  Take whatever steps are required to ensure that any non-detachable serial concentrator/terminal/device will never send unsolicited characters to the appliance.  This is referring to serial concentrators that might be configured to expect a terminal (DTE) on a serial port versus a DCE such as the Security Analytics.  Such devices could conceivably be programmed or configured to send serial data back to a device from which it receives such data (either by echoing characters or sending prompts telling users how to connect.  This kind of "prompting" or "echoing" can be seen by looking at the /var/log/messages file and searching for "SysRq" errors.