"VMware vSphere Lifecycle Manager encountered an unknown error. Please review the events and log files for more information" when scanning host for upgrade/update on 7.0.x or 8.0.x

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• "Host Upgrade" Compliance scan on 7.0.x or 8.0.x Update Manager fails with error "VMware vSphere Lifecycle Manager encountered an unknown error. Please review the events and log files for more information".


• In /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log:

<YYYY-MM-DD>T<time> error vmware-vum-server[2665284] [Originator@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore9ExceptionE(Connection reset by peer: The connection is terminated by the remote end with a reset packet. Usually, this is a sign of a network problem,  timeout, or service overload.)

• Above VUM log shows that the VUM received a reset from the ESXi while performing the Scan.
• In ESXi host, the /var/run/log/esxupdate.log will not have any error.
• In such scenario, Packet capture on both the VCSA and ESXi will show more data to understand the cause of failure.

Example: (ESXi packet capture sample)
3584    YYYY-MM-DD HH:MM:SS#.#.#.#    #.#.#.#    TCP    66    58940 → 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 WS=256
3392    YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    66    80 → 58940 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=512 SACK_PERM=1
3585    YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    60    58940 → 80 [ACK] Seq=1 Ack=1 Win=29440 Len=0

3586    YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    60    58940 → 80 [RST, ACK] Seq=1 Ack=1 Win=29440 Len=0

• Packet capture on ESXi shows that vCenter connected to ESXi on TCP port 80 and after TCP 3 way handshake vCenter sent a "Reset" packet to ESXi.

Example: (vCenter Server packet capture sample)

4029    YYYY-MM-DD HH:MM:SS   #.#.#.#    #.#.#.#    TCP    66    58940 → 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 WS=256
4030   YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    66    80 → 58940 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=512 SACK_PERM=1
4031    YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    54    58940 → 80 [ACK] Seq=1 Ack=1 Win=29440 Len=0

4033    YYYY-MM-DD HH:MM:SS    #.#.#.#    #.#.#.#    TCP    60    80 → 58940 [RST, ACK] Seq=1 Ack=140 Win=58880 Len=0

• Packet capture on vCenter shows that vCenter connected to ESXi on TCP port 80 and after TCP 3 way handshake ESXi sent a "Reset" packet to vCenter.
• Such behavior can be seen if there is a Firewall acting as an intermediate device closing session (Sending Reset) on both ends.

Environment

VMware vSphere ESXi 7.x
VMware vSphere vCenter 7.x

VMware vSphere ESXi 8.x
VMware vSphere vCenter 8.x

Cause

Firewall between ESXi and vCenter has a specific rule to block data transfer on TCP port 80.

Resolution

To resolve this issue remove the Firewall policy between the vCenter and ESXi on Port 80.