NVDS to VDS migration API fails at precheck stage without error description
search cancel

NVDS to VDS migration API fails at precheck stage without error description

book

Article ID: 378679

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

On VMware NSX-T 3.x, during a NVDS to VDS migration, the first step is to get a pre-check of the environment.

This can be done through:

  • NSX Manager UI
  • NSX Manager CLI
  • API call to NSX Manager

Reference to the NVDS to VDS migration guide: https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.2/administration/GUID-1039A36F-F55E-4A0A-B6C6-2C383F4A716D.html 

However, this precheck may return the message "Precheck status : failed" when using CLI or API methods. A similar error message is displayed when using the NSX UI for a pre-check. The common symptoms are the precheck failure return without any details. This prevents the migration with no impact to existing infrastructures. 

Similar log message may be in /var/log/proton/nsxapi

/var/log/proton/nsxapi

2024-08-03T16:14:52.644Z  WARN NvdsUpgradeTaskExecutor1 VcUtils 3790316 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] ConnectException occurred
java.net.ConnectException: Connection timed out (Connection timed out)

2024-08-03T16:14:52.644Z  WARN NvdsUpgradeTaskExecutor1 VcConnectionImpl 3790316 FABRIC [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] error while doing login to cm <compute manager>
java.net.ConnectException: Connection timed out (Connection timed out)

2024-08-03T16:14:52.644Z  WARN NvdsUpgradeTaskExecutor1 CmObjectFinderImpl 3790316 FABRIC [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Error occurred while getting host hardware network info
java.net.ConnectException: Connection timed out (Connection timed out)

Checking the following command in NSX manager Bash shell (root user):

uname -a

The kernel version from above command is =< 5.15.92 AND

cat /proc/sys/kernel/grsecurity/ip_blackhole

The above command returns ip_blackhole feature as 1 (turned on). 

 

Environment

VMware NSX-T 3.x

Cause

This is caused by a software issue within NSX-T.

NSX-T Manager Appliances running kernel version 5.15.92 with the ip_blackhole feature enabled may reject certain traffic flows, causing TCP timeout to occur.

Resolution

This issue is resolved in NSX-T 3.2.4 and newer. Customers impacted by this should upgrade to this version or a later version.

 


Workaround:

  • SSH into any NSX manager node as admin
    • admin>st en
    • root#: echo 'echo 0 > /proc/sys/kernel/grsecurity/ip_blackhole' >> /opt/vmware/nsx-node-api/bin/set_params.sh
  • Do the following command to disable ip_blackhole in runtime (rebootless).
    • root#: echo 0 > /proc/sys/kernel/grsecurity/ip_blackhole
  • Confirm that the ip_blackhole feature is turned off in runtime with following command returns "0" (turned off):
    • cat /proc/sys/kernel/grsecurity/ip_blackhole
  • Repeat the above steps for all NSX manager nodes
  • Retry the NVDS to VDS migration pre-check step

 

 

Powered by Wolken Software