When using VMware vShield App Firewall, virtual machines fail to connect to the vSwitch/vDS/network with the error: Failed to connect virtual device Ethernet0

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Ethernet0 fails to connect to the vSwitch/vDS
• In the vmware.log file for the affected virtual machine, you see entries similar to:
  
  vcpu-0| Msg_Post: Error
vcpu-0| [msg.ethernet.e1000.openFailed] Failed to connect ethernet0.
  
  The error in the vmware.log file may differ depending on the network driver used in the virtual machine, such as:
  
  vcpu-0| Msg_Post: Error
vcpu-0| [msg.ethernet.<network_driver>.openFailed] Failed to initialize ethernet0.
  
  
• In the hostd.log file on the ESXi host where the affected virtual machine is running, you see entries similar to:
  
  [F4DDDB90 info 'vm:/vmfs/volumes/4bdef87a-########-####-##########ac/DomainController/DomainController.vmx'] Question info: Failed to connect ethernet0., Id: 2 : Type : 3, Default: 0, Number of options: 1
[F4F88B90 verbose 'vm:/vmfs/volumes/4bdef87a-########-####-##########ac/DomainController/DomainController.vmx'] Retrieved current VM state from foundry 4, 8
[F4F88B90 warning 'vm:/vmfs/volumes/4bdef87a-########-####-##########ac/DomainController/DomainController.vmx'] Received a duplicate transition from foundry: 8, 1
[F4DDDB90 verbose 'Vmsvc'] VixVM_AnswerMessage returned 0
[F4DDDB90 info 'ha-eventmgr'] Event 235 : Message on DomainController on host.example.com in ha-datacenter: Failed to connect ethernet0.
[F4DDDB90 verbose 'vm:/vmfs/volumes/4bdef87a-########-####-##########ac/DomainController/DomainController.vmx'] Auto-answered question Failed to connect ethernet0.

[576C4B90 verbose 'vm:/vmfs/volumes/5044aaba-########-####-##########6b/MyVM01/MyVM01.vmx '] Handling message _vmx2: Failed to connect ethernet0. -->
[576C4B90 warning 'Locale'] FormatField: Invalid (vim.vm.Message.1)
[57189B90 info 'vm:/vmfs/volumes/5044aaba-########-####-##########6b/MyVM01/MyVM01.vmx'] Disconnect check in progress.
[576C4B90 warning 'Locale'] FormatField: Invalid (vim.vm.Message.1)
[576C4B90 info 'ha-eventmgr'] Event 62 : Error message on MyVM01 on VMHost1 in ha-datacenter: Failed to connect ethernet0. -->
  
  
• The Connected option does not remain selected for the virtual network adapter in the settings for the virtual machine.
• The virtual machine is manually removed (powered off, removed from inventory) from a host being managed by vShield App Firewall and then manually registered (browse datastore, add vmx to inventory) on a host that is not managed by vShield App Firewall.
• When connecting a virtual machine to a vSwitch that no longer has vShield App Firewall installed, you see an entry similar to this in the /var/log/vmkernel log file:
  
  vmkernel: 0:04:21:50.355 cpu7:4359)DVFilter: 2373: Could not find filter 'vshield-dvfilter-module'.

Environment

VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Installable
VMware ESX 4.0.x
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.0
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Embedded
VMware ESX 4.1.x
VMware vShield App 5.0.x

Cause

This issue can occur due to an incomplete installation or improper configuration of vShield App Firewall on the cluster.

Resolution

To work around this issue, add the affected virtual machine(s) to the vShield Manager App Firewall Exclusion list.

Alternatively, remove left over information from the failed installation or improperly configured vShield App Firewall from the virtual machine's .vmx configuration file.

To remove vShield related changes to the network from the .vmx file:

1. Power off the affected virtual machine.
2. Remove the virtual machine from inventory.
3. Open the virtual machine's .vmx file using a text editor.
4. Remove these lines (most likely at the end of the file):
   
   ethernet0.filter0.name = "vshield_filter_module_name"
ethernet0.filter0.name = "uuid=uuid_number"
   
   The lines may also appear as:
   
   ethernet0.filter0.name = "vshield_filter_module_name"
ethernet0.filter0.param1 = "uuid=uuid_number"
   
   Note: These lines are created during installation of vShield zones, and are recreated when it is installed again.
   
   
5. Register the virtual machine on the ESXi/ESX host.

Notes:

• The alternate solution only works if the VMware vShield App Firewall is completely un-installed. Otherwise, vShield Manager will re-add the lines again instantly.
• If you see these symptoms and removing the filter lines resolves the issue, but you are not using vShield App Firewall, you may be experiencing a known issue with the dvfilter kernel module not loading. For more information, see Booting the ESXi/ESX host fails with the error: initialization of module dvfilter failed (1031044).

Additional Information

To uninstall a vShield Zones instance:

1. Log in to the vSphere Client.
2. Select the ESXi/ESX host in the inventory.
3. Migrate all the virtual machines or power them off, as uninstalling vShield zones will put the host into maintenance mode.
4. Click the vShield tab.
5. Click Uninstall for the vShield Zones service.
6. Reboot the ESXi/ESX host.