Unable to login to OneClick Server, with a "No SpectroSERVER Available" error

Products

• Network Observability
• Spectrum

Issue/Introduction

Unable to login to OneClick (OC) Server, with a "No SpectroSERVER Available" error shown in the browser. 

Restarting both Tomcat & WebTomcat on the OC server made no difference, when attempting to login, it continues to return a "No SpectroSERVER Available" error.

Environment

DX NetOps Spectrum all currently supported releases

Cause

Cannot connect to the MLS with an error connecting to MLS landscape shown in catalina.out.

Checking for connections on MLS (192.168.0.2) to this OC server (192.168.0.1), port 14001 is shown in a permanent CLOSE_WAIT state so preventing any new connection to this OC:

# netstat -apn | grep 14001 | grep 192.168.0.1
tcp     13    0  192.168.0.2:35412     192.168.0.1:14001     CLOSE_WAIT    8342/SpectroSERVER

Port 14001 is used for CORBA communications between the SpectroSERVER to the OC server.

The CLOSE_WAIT state indicates that the remote end of the connection, in this case - the OC server, has sent a FIN packet to shutdown the connection, but the SpectroSERVER is still holding the socket open, waiting to perform its own shutdown or close but has not yet been able to.

Resolution

Stop the SpectroSERVER (SS) process on the MLS so as to get the OS to release the 14001 port. If force stopping the process, then the SSdb will need to be restored from backup.

Once stopped, the OS will flush the stuck port from the system and restore connections to the OC when SS is restarted.