Error: "java.lang.NullPointerException" when updating virtual machine properties in VMware Cloud Director.

Products

• VMware Cloud Director
• VMware Telco Cloud Infrastructure
• VMware Telco Cloud Platform

Issue/Introduction

• Updating Properties of a VM returns: 

class java.lang.NullPointerException

• Creating a new VM in the same Organization VDC(OVDC) fails with error

class java.lang.NullPointerException

• In the /opt/vmware/vcloud-director/logs/vcloud-container-debug.log file,you see

Error executing backend call. | requestId=########-####-####-####-############,
...
java.lang.NullPointerException
        at com.vmware.ssdc.backend.services.impl.VAppManagerImpl.getClusterMoref(VAppManagerImpl.java:4193)
        at com.vmware.ssdc.backend.services.impl.VAppManagerImpl.createUpdateVmParams(VAppManagerImpl.java:3501)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

Environment

VCD:  9.x, 10.3.3, 10.x

TCP: 5

TCI: 3

Cause

This issue occurs when there is an inconsistency in the VMware Cloud Director(VCD) database with the Resource Pool reference which is the 'getClustermoref' mentioned in the log entries.

Resolution

Note: This procedure modifies the database. Ensure to take a backup of the database before proceeding.

1. Take a backup of the Cloud Director database, for more details see Backup and Restore of VMware Cloud Director Appliance in the Cloud Director documentation.
2. SSH to the Primary Cloud Director Appliance and log in as root.
3. Connect to the Cloud Director database: 
   
   sudo -i -u postgres psql vcloud
   
   (or)
   
   db

For every OVDC you have, you will have at least one Resource Pool associated.
If the OVDC is comprised of more than one entry, you will have a row in the vrp_rp table for each Resource Pool.
The org_prov_vdc table has a sub_rp_moref entry which should map to the moref of the Resource Pool located under the Primary Cluster.

Issues with Resource Pools happen under the following conditions:

1. Resource Pool Moref does not map between vCenter and vCloud Director (resource pool recreated in vCenter)
2. Resource Pool deleted in vCenter

Step 1: Validate the Resource Pools Associated with the Org vDC

1. select id from vrp where name like '%<OVDC name>%'
2. select * from vrp_rp where vrp_id = <ID from the above step>

Note: If the above command returns 2 entries, verify in the VCD UI if the same number of resource pools are seen for the Org vDC

Step 2: Validate from the vCenter MOB that the Resource Pool object exists in vCenter.

1. In a web browser, navigate to http://vCenter_Server_name_or_IP/mob. (vCenter_Server_name_or_IP/mob is the name of your vCenter Server or its IP address).
2. Enter the SSO admin user credentials.
3. Click Content 
4. In the URL, remove the characters after moid= and place the values noted from the sub_rp_moref value of the vrp_rp table ie. moid=resgroup-1234
5. If the object exists in vCenter, the details are displayed. Otherwise "This page can't be found" is displayed.

Once validated, you will know under which categories this issue falls into.


Resource Pool Moref does not map between vCenter and Cloud Director (resource pool recreated in vCenter)
Depending on the number of Resource Pools, it may require updates to some or all of the related entries.
sub_rp_moref - org_prov_vdc, vrp_rp

update org_prov_vdc set sub_rp_moref = 'value in vCenter' where sub_rp_moref = 'old value'
update vrp_rp set sub_rp_moref = 'value in vCenter' where id = '<vrp.id from earlier>' and sub_rp_moref = 'old value'

Resource Pool deleted in vCenter

1. Recreate the Resource Pool in vCenter, and link up the values in org_prov_vdc and vrp, where sub_rp_moref is the Resource Pool moref as per the previous steps.
2. The customer can then decide if they want to keep or delete the OVDC

 

Once the Resource Pool references have been cleaned up in the Database, you should now be able to modify the VM. 
VCD Tenant UI > Applications > Virtual Machines > Right-click Virtual machine > Properties > Update the required parameters