NSX-T lost all the tags of VMs
Article ID: 322513
Updated On:
Products
Issue/Introduction
VMs lose tags once they are unregistered and registered after 30 minutes or VM is showing as inaccessible for longer than 30 minutes.
Relevant logs to look for:
For VMware NSX 3.1.x and below, look in: nsxapi.log
For VMware NSX 3.2.x and above look in: cm-inventory.log
2023-05-21T19:12:28.279Z INFO inventoryTasksScheduler-2 CleanupHandler - FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully deleted data for DeletedObjectId 50377b65-79bf-b062-5544-53fce12d7e74 of type VM
2023-05-21T19:12:30.568Z INFO inventoryTasksScheduler-2 CleanupHandler - FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully deleted data for DeletedObjectId 5037af0c-df6e-9f3e-4a23-2f0d79908552 of type VM
2023-05-21T19:14:28.309Z INFO inventoryTasksScheduler-2 CleanupHandler - FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Successfully deleted data for DeletedObjectId 50375893-a5de-5395-9cc6-0adf65225603 of type VM
Following log line indicates that 30 min times is triggered for VM with external id 50122b18-89fd-d37c-5ffb-b3e4196648a0:
-----------------------
2022-07-03T08:15:02.300Z INFO inventoryTasksScheduler1 VirtualMachineServiceImpl 4257 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="cm-inventory"] Marked VMContainer 50122b18-89fd-d37c-5ffb-b3e4196648a0 as deleted with timestamp 1656836102300
-----------------------
Following log line indicates that 30 min time limit is reached and the VM with external id 50122b18-89fd-d37c-5ffb-b3e4196648a0 is deleted:
-----------------------
2022-07-03T08:46:03.837Z INFO inventoryTasksScheduler10 CleanupHandler 4257 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="cm-inventory"] Successfully deleted data for DeletedObjectId 50122b18-89fd-d37c-5ffb-b3e4196648a0 of type VM
-----------------------
Environment
Cause
Host has reported with instance UUID for VMs after more than 30 mins. So the VMs got deleted and tags were lost.
- This is not a Bug & it is expected behavior.
- If a VM id marked for deletion from a host and if that VM is not claimed by any host within next 30-minute timeframe
then the VMs will be removed from the NSXT inventory, resulting in the loss of their NSXT tags.
Reference guide
- An example of expected behavior:
- Before the outage, the host sync with NSXT & vCenter reported 10 VMs.
- After an outage of multiple hours, when the host comes up & sync up with NSXT, if it has 9 VMs, then it will send a
full sync to NSXT. As it has only 9 VMs compared to the previous update of 10 VMs, NSXT will mark missing one VM for deletion; it will wait for 30 min to see if any host claims mark for deleted VM, if no other host claims that VM in
next 30 mins then NSXT will remove it from the inventory.
- If the VM is registered on a host after 30 minutes, it is treated as a new VM, and tags must be reapplied.
Resolution
Workaround:
In the scenario of a Virtual Machine losing its NSX-T tag(s) there are two workaround options:
- The NSX-T tags need to be manually applied to the affected VMs to ensure proper tagging and alignment with the network and security policies.
- Tag's can also be restored by restoring the NSX-T Manager backup (Restore a Backup (vmware.com)).
Additional Information
Impact/Risks:
VMs lost communication due to DFW rules not applying after losing NSX-T tags.
Feedback
