NSX Upgrade in SDDC Manager Upgrade fails due to VxRail API Timeout Issue

Products

• VMware SDDC Manager

Issue/Introduction

• NSX Upgrade failing on SDDC Manager after 20 mins or a short period of time 
• /var/log/vmware/vcf/lcm/lcm-debug.log or /var/log/vmware/vcf/lcm.log has the following entries: 

  INFO [vcf_lcm,0000000000000000,0000,upgradeId=#######-####-####-####-##########,resourceType=NSX_T_PARALLEL_CLUSTER,resourceId=<nsx_fqdn>:_ParallelClusterUpgradeElement,bundleElementId=#######-####-####-####-##########]
  [o.a.h.c.h.i.c.HttpRequestRetryExec,Upgrade-7] Recoverable I/O exception
  (java.net.SocketTimeoutException) caught when processing request to
  {s}->https://<vxrail_manager_fqdn>:443
  
  
  ERROR [vcf_lcm,0000000000000000,0000,upgradeId=#######-####-####-####-##########,resourceType=NSX_T_PARALLEL_CLUSTER,resourceId=<nsx_fqdn>:_ParallelClusterUpgradeElement,bundleElementId=#######-####-####-####-##########] [c.v.e.s.l.p.i.nsxt.NsxtUpgradeUtil,Upgrade-16] Unhandled exception during NSX component upgrade:
  java.net.SocketTimeoutException: Read timed out
          at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:283)
          at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:309)
          at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
          at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
          at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
  
  
  

 

Cause

VxRail system info API takes too long to respond

 

Resolution

• Take SSH to SDDC Manager VM as VCF user and switch to root (su root)
• Run the following command to test the time for the VxRail system info API to respond:

  time curl -k --user "[e-mail]:<password>" \
    --request GET "https://<VxRail_IP_address>/rest/vxm/v1/system"
  

•  If the above output shows more than 2 minutes to run the API, engage Dell to resolve the issue with VxRail Manager(s). Response in the logs should return "HTTP Status 200 - OK"