Unable to connect to vCenter Server due to port blocking
search cancel

Unable to connect to vCenter Server due to port blocking

book

Article ID: 418965

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

Symptoms:

  • Unable to connect to vSphere Client
  • Unable to connect to VAMI
  • Unable to connect to vCSA via SSH
  • Unable to ping vCSA

*However, the vCenter Server service has started normally.

Running: 
applmgmt lookupsvc lwsmd observability observability-vapi pschealth vc-wsla-broker vlcm vmafd vmcad vmdird vmware-analytics vmware-certificateauthority vmware-certificatemanagement vmware-cis-license vmware-content-library vmware-eam vmware-envoy vmware-envoy-hgw vmware-envoy-sidecar vmware-hvc vmware-infraprofile vmware-perfcharts vmware-pod vmware-postgres-archiver vmware-rhttpproxy vmware-sps vmware-stsd vmware-topologysvc vmware-trustmanagement vmware-updatemgr vmware-vapi-endpoint vmware-vdtc vmware-vmon vmware-vpostgres vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-ui vsstats vtsdb wcp
Stopped: 
vncam vmonapi vmware-imagebuilder vmware-netdumper vmware-rbd-watchdog vmware-vcha

  • Other virtual machines located on the same ESXi host and connected to the same DVPortGroup work without any issues.

  • MAC address changes are "Reject" in the GUI and CLI.
  • The vmkernel.log of the ESXi host where the vCSA is located contains the following output, and vc.eth0 is port blocked.

xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu3:8417767)Net: 2238: connected vc.eth0 eth0 to vDS, portID 0x4000112
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu3:8417767)Net: 3147: associated dvPort 69 with portID 0x4000112
::
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)Vmxnet3: 15304: Using default queue delivery for vmxnet3 for port 0x4000112
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: VSwitchPortEthFRPUpdateInt:5829: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]Unblock Port 67109138
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)NetPort: 3130: resuming traffic on DV port 69
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:237: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 requested mac address change to aa:aa:aa:aa:aa:aa on port 0x4000112, disallowed by vswitch policy
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:374: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 has policy violations on port 0x4000112. Port is blocked
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:237: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 requested mac address change to aa:aa:aa:aa:aa:aa on port 0x4000112, disallowed by vswitch policy
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:374: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 has policy violations on port 0x4000112. Port is blocked
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu0:2097211)NetPort: 708: Failed to acquire port non-exclusive lock 0x4000112[Failure].
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: VSwitchPortEthFRPUpdateInt:5829: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]Unblock Port 67109138
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:237: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 requested mac address change to aa:aa:aa:aa:aa:aa on port 0x4000112, disallowed by vswitch policy
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)cswitch: L2Sec_EnforcePortCompliance:374: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]client vc.eth0 has policy violations on port 0x4000112. Port is blocked
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)NetDVS: 9695: Property com.vmware.vswitch.port.macLearningStats, propType=0x20 (0x20), skip update for port 69
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu12:8420258)NetPort: 1618: enabled port 0x4000112 with mac aa:aa:aa:aa:aa:aa
::
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu7:8417767)NetPort: 1887: disabled port 0x4000112
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu7:8417767)Net: 3834: dissociate dvPort 69 from port 0x4000112
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu7:8417767)Net: 3841: disconnected client from port 0x4000112
::
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu42:8428770)Net: 2238: connected vc.eth0 eth0 to vDS, portID 0x4000113
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu42:8428770)Net: 3147: associated dvPort 69 with portID 0x4000113
::
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu41:8428834)NetPort: 3130: resuming traffic on DV port 69
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu41:8428834)cswitch: VSwitchPortEthFRPUpdateInt:5829: [nsx@6876 comp="nsx-esx" subcomp="vswitch"]Unblock Port 67109139
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu41:8428834)NetDVS: 9695: Property com.vmware.vswitch.port.macLearningStats, propType=0x20 (0x20), skip update for port 69
xxxx-xx-xxTxx:xx:xx.xxxZ In(182) vmkernel: cpu41:8428834)NetPort: 1618: enabled port 0x4000113 with mac bb:bb:bb:bb:bb:bb

  • Storage vMotion was performed immediately before.

xxxx-xx-xxTxx:xx:xx.xxxZ info vpxd[29376] [Originator@6876 sub=VmProv opID=lro-1-63d6ac32-13b-53-01-01] Local-VC Datastore Migrate of poweredOn VM 'vc' (vm-3003, ds:///vmfs/volumes/xxxxxxxxxxxxxx/vc/vc.vmx) on host-4023 (*.*.*.*) in pool resgroup-12 with ds ds:///vmfs/volumes/xxxxxxxxxxxxxx/ to host-4023 (*.*.*.*) in pool resgroup-12 with ds ds:///vmfs/volumes/xxxxxxxxxxxxxx/ with migId 4533054521255657627 as Operation: Local-VC_NonDRS_StoragevMotion
xxxx-xx-xxTxx:xx:xx.xxxZ info vpxd[29376] [Originator@6876 sub=VmCheck opID=lro-1-63d6ac32-13b-53-01-01] CompatCheck results: (vim.vm.check.Result) [
--> (vim.vm.check.Result) {
-->   vm = 'vim.VirtualMachine:bfe56e3e-dcc9-43f9-9a00-9b27a755a189:vm-3003',
--> host = 'vim.HostSystem:bfe56e3e-dcc9-43f9-9a00-9b27a755a189:host-4023',
--> }
--> ]

 

Environment

vCenter Server

ESXi

Cause

The vCSA guest OS initially used the MAC address "aa:aa:aa:aa:aa:aa."
However, after Storage vMotion, the MAC address was updated on the ESXi host (vmkernel), causing a discrepancy between the MAC address in the guest OS and the MAC address stored in the inventory of the ESXi host and vCenter Server.

Resolution

Restarting the vCSA will resolve the issue.

Additional Information

Check the vmware.log and vmx files on the vCSA to see if there are any MAC address conflicts.

If a conflict exists, restarting the vCSA will resolve it.

japanese KB: 418961

 

Below is an example of a conflict that can be seen in the vmware.log:

xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vmx - Ethernet0 MAC Address: aa:aa:aa:aa:aa:aa
xxxx-xx-xxTxx:xx:xx.xxxZ No(00) vmx - ConfigDB: Setting ethernet0.generatedAddress = "aa:aa:aa:aa:aa:aa"
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vmx - Ethernet0 MAC Address: aa:aa:aa:aa:aa:aa
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vcpu-0 - Ethernet0 MAC Address: aa:aa:aa:aa:aa:aa
xxxx-xx-xxTxx:xx:xx.xxxZ No(00) vmx HB-host-72@256558-39a7c236-bc-9b6d ConfigDB: Setting ethernet0.generatedAddress = "bb:bb:bb:bb:bb:bb"
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vcpu-1 - Ethernet0 MAC Address: aa:aa:aa:aa:aa:aa
xxxx-xx-xxTxx:xx:xx.xxxZ No(00) vcpu-1 - ConfigDB: Setting ethernet0.generatedAddress = "aa:aa:aa:aa:aa:aa"
xxxx-xx-xxTxx:xx:xx.xxxZ No(00) vmx HB-SpecSync-host-72@1005-add6702-4-9bef ConfigDB: Setting ethernet0.generatedAddress = "bb:bb:bb:bb:bb:bb"
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vcpu-14 - Ethernet0 MAC Address: bb:bb:bb:bb:bb:bb
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vmx - DICT ethernet0.generatedAddress = "bb:bb:bb:bb:bb:bb"
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vmx - Ethernet0 MAC Address: bb:bb:bb:bb:bb:bb
xxxx-xx-xxTxx:xx:xx.xxxZ In(05) vmx - Ethernet0 MAC Address: bb:bb:bb:bb:bb:bb

Powered by Wolken Software