NSX-T generates a large number of log messages for IDPS reporting

Products

• VMware vDefend Firewall

Issue/Introduction

Symptoms:

• You have IDPS enabled and may have URL filtering enabled.
• You receive a high number of log ERROR messages from vRLI or other log reporting tool relating to the IDPS reporting service:

[nsx@6876 comp="nsx-manager" errorCode="MP101" level="ERROR" subcomp="idps-reporting"] Closing channel because ChannelAckStreamTask(channel=ServerChannel(vmware.nsx.context.ids.IdsMpService, ########-####-####-####-########99e2), settings=version: 2#012supported_version: 1#012supported_version: 2#012supported_version: 3#012short_method_names_support: 1#012flow_control_support: 1#012trace_support: 1#012) leaked exception java.lang.NullPointerException#012

• A similar large number of log messages relating to the IDPS reporting service can also be seen in the NSX-T Manager logs idps.log:

INFO unix:securitydataservice:worker-0 AbstractConnection - - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="idps-reporting"] NettyConnection(NettyChannel(local=null, remote=var/run/vmware/appl-proxy/aph.sock), active=true) registering ServerChannel(vmware.nsx.context.ids.IdsMpService, ########-####-####-####-########60b2)
ERROR nsx-rpc:securitydataservice:user-executor-0 ChannelTask - - [nsx@6876 comp="nsx-manager" errorCode="MP101" level="ERROR" subcomp="idps-reporting"] Closing channel because ChannelAckStreamTask(channel=ServerChannel(vmware.nsx.context.ids.IdsMpService, ########-####-####-####-########60b2), settings=version: 2
supported_version: 1
supported_version: 2
supported_version: 3
short_method_names_support: 1
flow_control_support: 1
trace_support: 1
) leaked exception java.lang.NullPointerException
INFO nsx-rpc:securitydataservice:user-executor-0 NsxRpcChannel - - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="idps-reporting"] ServerChannel(vmware.nsx.context.ids.IdsMpService, ########-####-####-####-########60b2).doClose(closeStream=true, status=Status(code=UNKNOWN, msg=ChannelTask exception: java.lang.NullPointerException
))
INFO nsx-rpc:securitydataservice:user-executor-0 NsxRpcChannel - - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="idps-reporting"] ServerChannel(vmware.nsx.context.ids.IdsMpService, ########-####-####-####-########60b2) doCloseStream OutboundMessage(streamId=null payloadSize=0 streamControl=close_stream: true
close_status {
  code: UNKNOWN
  error_msg: "ChannelTask exception: java.lang.NullPointerException\n"
}
 frame=)

•  No workload impact is noticed due to these alerts.

Environment

VMware NSX-T Data Center
VMware NSX-T Data Center 3.x

Cause

This issue is due to a race condition in an internal process and how it handles connections from JVM processes to the internal APH (Appliance Proxy Hub).

Resolution

This issue is resolved in NSX-T 3.1.3

Workaround:
To mitigate this issue, please restart the IDPS reporting service.
You will need to log in as root on each manager and do the following:

/etc/init.d/idps-reporting-service restart  

If the issue persists after the service restart, you may need try a few times, as this is a race condition.