vRealize Network Insight shows upgrade pre-checks failure using vRSLCM 8.7.0 and 8.8.0

Products

• VMware Aria Operations for Networks
• VMware Aria Suite

Issue/Introduction

Purpose of this knowledge base article is to fix this issue using the workaround mentioned.

Symptoms:

vRSLCM calls api with stale data when upgrade is retried through VRSLCM and hence VRNI Upgrade fails in pre-checks 

Error message seen:

com.vmware.vrealize.lcm.plugin.core.vrni.common.exception.VRNIUpgradeCheckStatusException: Error occurred while checking upgrade pre-check status with IP - Pre-check message :

{ "msg" : "HBase or HDFS is unhealthy", "type" : "INFO", "status" : "FAIL", "id" : "HBaseHdfsHealthCheckTask", "title" : "HBase and HDFS Health Check", "consentMsg" : null}

Environment

VMware vRealize Suite Lifecycle Manager 8.8.x
VMware vRealize Suite Lifecycle Manager 8.7.x
VMware vRealize Network Insight 6.6.0

Cause

During upgrade of vRNI using vRLCM some of vRNI services if not running, upgrade request pre-check task fails and subsequent retries in vrslcm uses the stale data present in vRNI

Resolution

See Workaround section

Workaround:

Run below steps on platform1/single node 
1) Find pre-checks UUID, version number and upgrade type using below command:
sudo grep  "Start upgrade checks for id" /var/log/arkin/pre_checks.log | tail -1 | awk '{print $6 $8 $10}' | sed 's/,/ /g'

Example Output: bdddd00a-XXXX-XXXX-XXXX-XXXXXXXXXXXX 6.6.0.1648892975 OFFLINE

2) Trigger pre-checks using the output above in the below command:
sh /home/ubuntu/build-target/pre-upgrade/pre.sh <Output of 1st command>

Example Command:     
sh /home/ubuntu/build-target/pre-upgrade/pre.sh bdddd00a-XXXX-XXXX-XXXX-XXXXXXXXXXXX 6.6.0.1648892975 OFFLINE

3) Verify if pre-checks passed

Command:
sudo tail ~/logs/pre_checks.log
 
Output: last few lines of file(~/logs/pre_checks.log) should contain such lines:  

Checking if HDFS is healthy 
Checking if HBase is healthy 
Done upgrade checks for id bdddd00a-XXXX-XXXX-XXXX-XXXXXXXXXXXX, version 6.6.0.1648892975, upgradeType OFFLINE

Additional Information

Impact/Risks:

Setups which use vRLCM 8.7.0 and 8.8.0 and upgrade fails during reattempt due to pre-checks failure.