VMware vCloud Foundation password lookup on the VRM virtual machine fails with the error: "getIbPasswordTool() failed"
book
Article ID: 316910
calendar_today
Updated On:
Products
VMware Cloud Foundation
Issue/Introduction
Symptoms:
When attempting to perform the /opt/vmware/evosddc-support/sos --health-check command, you see results similar to:
Welcome to Supportability and Serviceability(SoS) utility! Health Check : /var/tmp/healthcheck-2017-12-10-14-04-31-3116 Health Check log : /var/tmp/healthcheck-2017-12-10-14-04-31-3116/sos.log Health check initiated on rack/s : [192.168.100.110] Skipping Health Check since could not get VRM CLI password Health check report for 192.168.100.110 : /var/tmp/healthcheck-2017-12-10-14-04-31-3116/rack-1 Health Check log : /var/tmp/healthcheck-2017-12-10-14-04-31-3116/sos.log Health Check completed successfully for : [] Operation failed for : [HEALTH-CHECK]
When attempting to perform the /home/vrack/bin/vrm-cli.sh --lookup-password command, you see this error:
ERROR: getIbPasswordTool() failed.
In the /home/vrack/vrm/logs/vrm-cli.log file, you see errors similar to:
2017-12-10 20:44:23.720 [main] ERROR [com.vmware.vrack.vrm.cli.VRMCli$Stdout] getIbPasswordTool() failed java.lang.NullPointerException: null at java.io.StringReader.<init>(Unknown Source) ~[na:1.7.0_141] at org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:636) ~[jackson-core-asl-1.9.9.jar:1.9.9] at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1870) ~[jackson-mapper-asl-1.9.9.jar:1.9.9] at com.vmware.vrack.vrm.cli.commands.helpers.HmsPasswordTool.getAllNodes(HmsPasswordTool.java:154) ~[vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.HmsPasswordTool.getSwitches(HmsPasswordTool.java:194) ~[vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.HmsIbPasswordTool.getSwitches(HmsIbPasswordTool.java:168) ~[vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.HmsIbPasswordTool.getSwitches(HmsIbPasswordTool.java:159) ~[vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.LookupInventory.populate(LookupInventory.java:174) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.LookupInventory.mergeHmsInventory(LookupInventory.java:112) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.helpers.LookupInventory.<init>(LookupInventory.java:76) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.VrmPropertiesManager.getIsvmWorkload(VrmPropertiesManager.java:262) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.VrmPropertiesManager.getIsvmAppliances(VrmPropertiesManager.java:239) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.LookupPassword.buildOutput(LookupPassword.java:72) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.LookupPassword.printOutput(LookupPassword.java:56) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.commands.LookupPassword.main(LookupPassword.java:47) [vrm-cli-2.1.3-RELEASE.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_141] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_141] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_141] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_141] at com.vmware.vrack.vrm.cli.VRMCli.invoke(VRMCli.java:311) [vrm-cli-2.1.3-RELEASE.jar:na] at com.vmware.vrack.vrm.cli.VRMCli.main(VRMCli.java:466) [vrm-cli-2.1.3-RELEASE.jar:na]
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware Cloud Foundation 2.1.x
Cause
This issue might occur when communicating with the Hardware Management Services on the Management Switch.
Resolution
To resolve this issue, VMware recommends to restart the Management Switch followed by the VRM Virtual Machine.
Open a console session or use SSH to connect to the VRM VM and log in as root.
Start an SSH session to the Management Switch and log in using the cumulus username and password.
On the Management Switch, issue this command using the root password to enter a root session:
su -
Issue this command:
reboot
When the Management Switch is up, issue this command on the VRM VM as root:
reboot
After the VM is up, wait approximately 10 minutes for all the services to fully start.