Linux Guest OS shows network interfaces re-arranged (mixed sequence) after adding new ones and reboot.

Products

• VMware vSphere ESXi

Issue/Introduction

When a new Network interface is added to a Linux VM, it will appear not as the last added interface after the VM is rebooted. Due to this, the MAC Address of the interfaces sequence are shuffling.

Here is a sample of the change of sequence inside the guest OS simulated in lab:

 

If there are any reference to the sequence of the interface instead of it's name (i.e. Interface 2 instead of ens192), since the MAC Address of that sequence will be different, the reference will not be able to find the same MAC Address.The sequence is not scrambled on the virtual hardware, but on the guest OS only.

The ethernet interfaces are correctly associated with the MAC Address with no changes on the sequence in the virtual hardware.

[root@ESXI_NAME:/vmfs/volumes/########-########-####-############/VM_NAME] grep eth *.vmx
ethernet0.allowGuestConnectionControl = "FALSE"
ethernet0.virtualDev = "vmxnet3"
ethernet0.networkName = "My_Lab_Network"
ethernet0.addressType = "vpx"
ethernet0.generatedAddress = "00:50:56:aa:74:13" (ens192)
ethernet0.uptCompatibility = "TRUE"
ethernet0.present = "TRUE"
ethernet0.pciSlotNumber = "192"
ethernet1.allowGuestConnectionControl = "FALSE"
ethernet1.virtualDev = "vmxnet3"
ethernet1.networkName = "My_Lab_Network"
ethernet1.addressType = "vpx"
ethernet1.generatedAddress = "00:50:56:aa:84:d6" (ens224)
ethernet1.uptCompatibility = "TRUE"
ethernet1.pciSlotNumber = "224"
ethernet1.present = "TRUE"
ethernet2.allowGuestConnectionControl = "FALSE"
ethernet2.virtualDev = "vmxnet3"
ethernet2.networkName = "My_Lab_Network"
ethernet2.addressType = "vpx"
ethernet2.generatedAddress = "00:50:56:aa:c6:a3" (ens256)
ethernet2.uptCompatibility = "TRUE"
ethernet2.pciSlotNumber = "256"
ethernet2.present = "TRUE"
ethernet5.allowGuestConnectionControl = "FALSE"
ethernet5.virtualDev = "vmxnet3"
ethernet5.networkName = "My_Lab_Network"
ethernet5.addressType = "vpx"
ethernet5.generatedAddress = "00:50:56:aa:ae:1c" (ens225)
ethernet5.uptCompatibility = "TRUE"
ethernet5.pciSlotNumber = "1248"
ethernet5.present = "TRUE"
ethernet3.allowGuestConnectionControl = "FALSE"
ethernet3.virtualDev = "vmxnet3"
ethernet3.networkName = "My_Lab_Network"
ethernet3.addressType = "vpx"
ethernet3.generatedAddress = "00:50:56:aa:d7:eb" (ens161)
ethernet3.uptCompatibility = "TRUE"
ethernet3.pciSlotNumber = "1184"
ethernet3.present = "TRUE"

Note that ens256 and ens225 were already on different sequence when ens161 was added.

 

Environment

vSphere ESXi 8.0.x

 

Cause

This happens because the guest OS orders the network interface sequence according to the PCI slot number. There are no issues on the hypervisor level, the virtual hardware is provided as expected to the guest OS.

ESXi will define the pciSlotNumber sequentially (i.e., if you remove the ethernet and add it again, the same pciSlotNumber will be assigned)

 

Resolution

Make sure the biddings made on the guest OS level are associated with the correct ethernet adapter.

Some guest OS will assign sequential names to the network adapters causing issues when bidding features to the sequence instead of the name of the interface. Therefore the guest OS vendor should be contacted for further assistance.

To understand the mapping of the pciSlotNumber, check the KB 311606 (Mapping PCI slot numbers to guest-visible PCI bus topology).