EMC M&R | Watch4net | ViPR SRM| VNX M&R: 'backend' will not start / java.net.BindException: Address already in use: JVM_Bind, throwing / APG Backend startup failed!

Products

• VMware

Issue/Introduction

Symptoms:


Backend service will not start
 

Backend service is stopped and fails to start:

manage-modules.sh service status backend
 * Checking 'backend Default'...                                    [ stopped ]
manage-modules.sh service start backend
 * Starting 'backend Default'...                                     [ failed ]

Backend logs shows one of the following errors:

APG/Backends/APG-Backend/Default/logs/cache-0-0.log

INFO  -- [2012-11-15 14:23:31 CAT] -- AbstractServer::configure(): Parsing telnet control interface server config file E:\APG\Backends\APG-Backend\Default\.\conf\telnetinterface.xml
INFO  -- [2012-11-15 14:23:31 CAT] -- CacheControlInterfaceManager::initialize(): Initializing control interface  'Telnet-Control'...
INFO  -- [2012-11-15 14:23:31 CAT] -- AbstractServer::init(): Binding telnet control interface server on 2001...
SEVERE  -- [2012-11-15 14:23:31 CAT] -- AbstractServer::init(): java.net.BindException: Address already in use: JVM_Bind, throwing...
SEVERE  -- [2012-11-15 14:23:31 CAT] -- Cache::start(): APG Backend startup failed!
com.watch4net.apg.v2.common.io.InOutException: Can't create and bind the telnet control interface server socket on port 2001

------------------------

SEVERE     -- [2013-07-17 11:26:24 EDT] -- Cache::start(): APG Backend startup failed!
com.watch4net.apg.v2.common.io.InOutException: Can't create and bind the socket collector interface server socket on port 2000
    at com.watch4net.apg.v2.common.io.AbstractServer.init(AbstractServer.java:368)
    at com.watch4net.apg.v2.cache.io.plugins.SocketCollectorInterface.init(SocketCollectorInterface.java:107)
    at com.watch4net.apg.v2.cache.io.CacheControlInterfaceManager.initialize(CacheControlInterfaceManager.java:102)
    at com.watch4net.apg.v2.cache.Cache.start(Cache.java:319)
    at com.watch4net.apg.v2.cache.Cache.serviceStart(Cache.java:798)
    at com.watch4net.apg.v2.cache.Cache.main(Cache.java:877)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.watch4net.apg.module.plugin.service.Bootstrap.main(Bootstrap.java:59)
    at com.watch4net.apg.module.plugin.service.ProcrunBootstrap.main(ProcrunBootstrap.java:80)
Caused by: java.net.BindException: Address already in use: JVM_Bind
    at java.net.PlainSocketImpl.socketBind(Native Method)
    at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
    at java.net.ServerSocket.bind(ServerSocket.java:328)
    at java.net.ServerSocket.bind(ServerSocket.java:286)
    at com.watch4net.apg.v2.common.io.AbstractServer.init(AbstractServer.java:364)

Environment

VMware Smart Assurance - Watch4Net/M&R

Cause

Another process on the host is using the same port 2000 or 2001 (in this case)
port 2000 is used by the primary backend socket interface & port 2001 is used by the primary backend telnet interface

See the ViPR SRM community space for a complete list of ports used

Resolution

Programs such as netstat, ps aux on linux or the Task Managers on Windows can help determine what program is using the port.

Linux:
netstat -aonp | grep <port>
For example:
netstat -aonp | grep 2001

Windows:
netstat -aon | findstr <port>
For example:
netstat -aon | findstr "2001"

The the case of ghost/zombie process terminating it or rebooting the host will resolve the issue; in the case of a conflict with a third party application and you choose to change the ports used by EMC M&R please contact EMC Customer Support and reference KB 87890.