Monitoring VMs using Veritas Tool fails on VMware vCenter

Products

• VMware vCenter Server

Issue/Introduction

•  Veritas Cluster Server (VCS) engine_A.log reports:
  yyyy/mm/dd hh:mm:ss VCS ERROR V-16-2-13027 (xxxxxx) Resource(vmdisk_dgsh01) - monitor procedure did not complete within the expected time.
  
  
  
• The VMwareDisks_A.log file on Veritas application report there is a problem with the VMwareDisks agent contacting the ESX server-
  
  yyyy/mm/dd hh:mm:ss VCS ERROR V-16-10061-22503 VMware Disks: ###_vmdk_1: monitor: Failed to login to ESX <ESXi Host name> with error 'SOAP_TCP_ERROR SOAP 1.1 fault: SOAP-ENV: Client [no subcode] "Operation now in progress"
Detail: connect failed in tcp_connect()
yyyy/mm/dd hh:mm:ss VCS DBG FFDC Generating FFDC for resource (###_vmdk_1) as monitor entry point reported UNKNOWN
yyyy/mm/dd hh:mm:ss VCS DBG_AGDEBUG Calling monitor for resource ###_vmdk_1
VCSAgType.C: call_monitor [1495]
yyyy/mm/dd hh:mm:ss VCS DBG 1 VMware Disks: ###_vmdk_1: monitor: No vCenter address configured for VMware HA
VMwareDisks.cpp: VMwareDisksGetArgs [1461]
yyyy/mm/dd hh:mm:ss VCS DBG 1 VMwareDisks: ###_vmdk_1:monitor: timeout is 40 VMwareDisks.cpp:Login [1752]
yyyy/mm/dd hh:mm:ss VCS DBG 1 VMware Disks: ###_vmdk_1:monitor: Failed to logout from ESX <ESXi Host name> with error 'Session not available' VMwareDisks.cpp: vmwaredisks_monitor [467]
yyyy/mm/dd hh:mm:ss VCS DBG_AGINFO Resource (###_vmdk_1) - monitor entry point exited with a confidence value 0. VCSAgType.C: call_monitor [1587]
yyyy/mm/dd hh:mm:ss VCS DBG_AGINFO ###_vmdk_1 reported state (Unknown) & conf_level (0)
VCSAgRes.C:call_entry_point [1603]
yyyy/mm/dd hh:mm:ss VCS DBG AGDEBUG Canceling timer for (###_vmdk_1) op (1608)
VCSAgTimer.C:_cancel [953]


•  vCenter logs (vpxd) report -
  vpxd-165.log:yyyy/mm/dd hh:mm:ss  info vpxd[06530] [Originator@6876 sub=AuthorizeManager] [Auth]: User <user-name>
vpxd-165.log:yyyy/mm/dd hh:mm:ss .123+07:00 error vpxd[06528] [Originator@6876 sub=User opID=1ae0a4b2] Failed to authenticate user <user-name>
vpxd-165.log:--> "user-name"

 

• Initiating a ping request from Veritas VM's Guest OS or Port connection request on port 443 works well to conclude no Network issue.
  
  
• Wireshark analysis of packets captured at the vCenter switch ports and the Veritas device’s NIC/VM switch port confirms that the Veritas device is initiating the TCP RSTs.

         Veritas VM -
         # pktcap-uw --switchport <switchport ID>  --ip <vCenter IP>  --capture VnicRx,VnicTx -o /vmfs/volumes/datastore-name/veritas-vm.pcap

     
        vCenter VM -
    # pktcap-uw --switchport <switchport ID> --ip <Veritas IP>  --capture VnicRx,VnicTx -o /vmfs/volumes//datastore-name/vCenter.pcap   

 

         

Environment

VMware vSphere

Cause

Veritas Application is aborting the TCP connection by sending a TCP RST packet.

Resolution

This is not a VMware issue. The TCP resets originate from the Veritas machine, please open a support case with Veritas for further assistance.

Additional Information

Should further information from Broadcom be required, please open a case with Broadcom Support.