NSX Controller disk becomes full and disconnects

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

One of the node disk that is part of an NSX Cluster gets full and gets into a disconnected state.

Note: For additional symptoms and log entries, see the Additional Information section.

Environment

VMware NSX for vSphere 6.0.x
VMware NSX for vSphere 6.2.x
VMware NSX for vSphere 6.1.x

Cause

This issue occurs because, by default, a group of logs (including mail.log) on the controller rotates every week. This configuration stores the logs on the controller disk for a week before the log rotates.

Resolution

This issue is resolved in VMware NSX for vSphere 6.2.4.

To work around this issue if you do not want to upgrade, file a support request with Broadcom Support and quote this Knowledge Base article ID (341196) in the problem description. 

Additional Information

You experience these additional symptoms:

• When running the show status command on NSX controller console, you see entries similar to:
  
  Version: 4.0.6 (Build 46427)
  Current Time: Thu, 19 May 2016 08:30:37 +0000
  Uptime: 1 day 16 hours 26 minutes 28 seconds
  Load Average: 0.28, 0.20, 0.11
  Memory Usage: 3918752 kB (Total), 426272 kB (Free)
  Disk Usage:
  Filesystem 1K-blocks Used Available Use% Mounted on
  /dev/mapper/nvp-root 16263588 16263588 0 100% /
  udev 1950236 4 1950232 1% /dev
  /dev/sda1 233191 37269 183481 17% /boot
  
  
• NSX Controller shows as disconnected in vCenter Web Client > NSX Manager Plugin > Installation view.