- Upgrade is stuck in UI on step 13 out of 37.
- In VMware Aria operation on Network, we see below error:
Failed to execute sudo -u ubuntu /home/ubuntu/build-target/launcher/upgrade-services.sh /tmp/upgrade_zookeeper_1725688792fg5lpyna.tar.gz /home/ubuntu/build-target/zookeeper ubuntu /home/ubuntu/build-target/deployment zookeeper 1725688792 /home/ubuntu/build-target/zookeeper >> /var/log/arkin/centralized_upgrade.log 2>&1
- ./run_all.sh sudo /home/ubuntu/check-service-health.sh -p -d shows below services are not running or not healthy
ubuntu@aria-networks-platform:~$ ./run_all.sh sudo /home/ubuntu/check-service-health.sh -p -d
This is not a cluster environment. Running the command on localhost only...
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
ElasticSearch is running and healthy.
ElasticSearch statistics:
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-14:09:27
HMaster is running but not healthy.
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-14:08:40
Problem: HRegionServer javaservice is not running.
QuorumPeerMain is running and healthy.
QuorumPeerMain statistics:
Uptime:3-14:08:50
ResourceManager is running and healthy.
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-14:08:43
NodeManager is running but not healthy.
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-14:08:43
Launcher is running
Uptime:01:27
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
VIPService is running and healthy.
VIPService statistics:
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-14:08:34
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
DatabusGateway is running and healthy.
DatabusGateway statistics:
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:3-13:47:51
Problem: FlinkContainer javaservice is not running.
SaasListener is running
Uptime:3-13:48:07
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Kafka is running and healthy.
Kafka statistics:
Python FIPS exe: /usr/local/bin/python3.6 not found!
Use prev python3 in the event proper python3 FIPS is not found (upgrade scenarios
Previous python FIPS: /usr/local/bin/python3.6 is also not here as well! Will resort to base python3 to ensure we can continue...
NOTE: Setting LD_LIBRARY_PATH to old FIPS for any potential python3.6 calls: /opt/openssl-fips/openssl-1.0.2y/lib64
Uptime:12:57
Restapilayer is running and healthy.
Restapilayer statistics:
Uptime:3-13:49:10
Problem: TSDB javaservice is not running.
Nginx is running and healthy.
Nginx statistics:
Uptime:3-13:47:25
ExpressJSApp is running
Uptime:3-13:52:02
NTPSEC is running and healthy.
Uptime:3-13:56:03
FoundationDB is running and healthy.
FoundationDB statistics:
Uptime:3-13:46:40
3-13:46:40
VMware Aria operation on Network 6.12
Zookeeper.conf file is missing from platform VM.
sudo tail -f /logs/chef-run.log shows entries of missing Zookeeper.conf files
The issue was resolved by removing the binary version markers. If you encounter this issue, please contact Broadcom Support via an SR and mention this KB article, more information Creating and managing Broadcom support cases