VNA operational status down in PM

book

Article ID: 105931

calendar_today

Updated On:

Products

CA Infrastructure Management CA Infrastructure Management CA Performance Management - Usage and Administration CA Performance Management - Data Polling

Issue/Introduction

We have VNA 3.5 running in our environment. We ran out of disk space
on the VNA VM and the operational status of VNA Gateway in PM changed to down.
We increased the disk space on VM and ran service start 'wildfly', but we still see the same error. 
The following exception was seen in /opt/CA/VNA/wildfly/standalone/log/server.log

Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago.
The driver has not received any packets from the server.
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
    at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
    at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:341)
    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2203)
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2236)
    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2035)
    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790)
    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)

Cause

mysql.sock.lock with nothing in it was left behind after the reboot

When we attempted to start mysql we saw the following:

ERROR! MySQL is not running, but lock file (/var/lock/subsys/mysql) exists
# service mysql start
Starting MySQL... ERROR! The server quit without updating PID file (/opt/CA/MySql/data/<hostname>.pid).

When looking at the /opt/CA/MySql/data/<hostname>.err file we saw the following
 Server socket created on IP: '::'.
[ERROR] Unix socket lock file is empty /opt/CA/MySql/data/mysql.sock.lock.
[ERROR] Unable to setup unix socket lock file.
[ERROR] Aborting

Environment

CA VNA 3.5

Resolution

To resolve we did the following:

1. Removed the /var/lock/subsys/mysql
2. Removed /opt/CA/MySql/data/mysql.sock.lock
3. service mysql start - which started mysql
4. service wildfly stop / start to restart wildfly successfully