Pivotal Bosh provision virtual machines fail randomly on NSX-T

Products

• VMware NSX Networking

Issue/Introduction

Symptoms:

• Newly created virtual machines fail to show up on NSX-T.
• In the nsxapi.log file in proton of the NSX Manager, you see entries similar to:  
  
  MissingVmInventoryException at com.vmware.nsx.management.inventory.discovery.VniMessageProcessorImpl.addVni_aroundBody0.

ERROR task-executor-12 VniMessageProcessorImpl - FABRIC [nsx@6876 comp="nsx-manager" errorCode="MP7021" subcomp="manager"] Attempt to create vni failed. Tried creating after vni was not found in update : id: "50015b39-9b21-6d7e-90a1-e6833934a64a-4000"

Environment

VMware NSX-T 1.0.x
VMware NSX-T

Cause

This issue occurs when a very quick vMotion or cloning+deletion happening continuously can lead to the scenario where the virtual machines are getting deleted before they are attached to logical switches.

Resolution

This issue is resolved in VMware NSX-T Data Center 2.2, available at VMware Downloads.

Workaround:
To work around this issue, reduce the inventory sync processing.

1. Log in to the NSX Manager appliance with the root user.
2. Update the repositories:
   
   # apt-get update
    
3. Install zip & unzip:
   
   # apt-get install zip unzip
          
4. Modify the inventory-config.properties configuration file:
   
   # mkdir inventory
   # cd inventory
   ~/inventory# cp /opt/vmware/proton-tomcat/webapps/nsxapi/WEB-INF/lib/inventory-mgmt-1.0.jar .
   ~/inventory# unzip inventory-mgmt-1.0.jar
   ~/inventory# ls -l
       builddef.lst com
       inventory-config.properties
       inventory-mgmt-1.0.jar META-INF
   ~/inventory# vim inventory-config.properties
       Change "inventory.discovery.message.processing.batch.size=10" to "inventory.discovery.message.processing.batch.size=1"
       save and exit (:wq)
      
5. Clean up and replace original inventory-mgmt-1.0.jar with modified version:
   
   ~/inventory# rm inventory-mgmt-1.0.jar
   ~/inventory# zip -r inventory-mgmt-1.0.jar .
   ~/inventory# cp inventory-mgmt-1.0.jar /opt/vmware/proton-tomcat/webapps/nsxapi/WEB-INF/lib/
    
6. Restart proton:
   
   ~/inventory# /etc/init.d/proton restart