SDDC Manager UI is unable to display the physical inventory

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• The VMware Cloud Foundation 2.2.x SDDC Manager UI is unable to display the physical inventory
• The /opt/vmware/sddc-support/sos utility returns failures when collecting logs/configuration data.
• The /home/vrack/bin/lookup-passwords command returns errors similar to the following:

ERROR: no content supplied
ERROR: curl failed: HTTP error code 400 or above.
ERROR: lookup-passwords command failed
Usage: Decrypt [content]

• Messages similar to the following are present in the /opt/vmware/cassandra/apache-cassandra-2.2.4/logs/system.log file:

Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: /opt/vmware/cassandra/apache-cassandra-2.2.4/bin/../data/data/vrmkeyspace/esxianddomain-13956920a44f11e7a69b01adb9a851d5/la-1-big-Data.db

Cause

This issue can occur when there is a corruption one or more files used by the Cassandra database.

Resolution

This is a known issue affecting VMware Cloud Foundation 2.2.x. There is currently no resolution.

Workaround:
Note: Ensure that there is a current backup or snapshot of the SDDC Manager Controller virtual machine prior to proceeding.

1. Log in to the SDDC Manager Controller virtual machine as the root user.
2. Issue the following commands to stop all VMware Cloud Foundation related services:

systemctl stop scs
systemctl stop vcfmanager
systemctl stop hms
systemctl stop lcm
systemctl stop zookeeper
systemctl stop cassandra

3. Use the nodetool command to correct the issues with the corrupted file(s):

nodetool scrub

4. Issue the following commands to start all VMware Cloud Foundation related services:

systemctl start scs
systemctl start cassandra
systemctl start zookeeper
systemctl start vcfmanager
systemctl start hms
systemctl start lcm

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box.

Impact/Risks:
The workaround provided can cause irreparable damage to the Cassandra database if not done correctly. The workaround may also not correct the issue.