Hot added disks go missing after a power off or migration of the virtual machine

Products

• VMware vSphere ESXi

Issue/Introduction

Virtual machine configuration updates like hot adding CPU/Memory or disks, resource allocation changes may be lost after a power off or migration.

Cause

Virtual machine configuration is set to "Read Only". 

Read only state for the configuration is set for brief period during VMotion of the virtual machine. This state is expected to be reset back once the migration is complete. In cases when there is a catastrophic failure during the migration, a partial execution of migration workflow may result in virtual machine configuration permanently ending up in read only state. 

Confirming if the virtual machine configuration is set to read only. 

1. Login to ESXi Server where the VM is registered using remote SSH or local TSM shell 
2. Locate the virtual machine configuration path using command,

   vim-cmd vmsvc/getallvms

3. Check if the config paramter responsible for read only state present in the file

   grep config.readOnly /vmfs/volume/<DATASTORE_NAME>/<VM_HOME_FOLDER>/<VM_NAME>.vmx

4. If set you will see an output similar to

   config.readOnly = "TRUE" 

Resolution

The configuration entry causing the read only state of the virtual machine need to be removed. 

Caution: Powering off the virtual machine in this state will result in changes done to the configuration since the last power on to be lost. Make sure you have recoding of the all the changes. Additionally make sure you have a valid/usable backup of the virtual machine.

1. Make a note off all the changes to the virtual machine configuration since the last power on
   1. Hot added virtual hardware devices, resource allocation changes etc..
2. Power off the virtual machine
3. Login to ESXi Server where the VM is registered using remote SSH or local TSM shell 
4. Locate the virtual machine configuration path using command, 

   vim-cmd vmsvc/getallvms

5. Make a backup copy of the virtual machine configuration

   cp /vmfs/volume/<DATASTORE_NAME>/<VM_HOME_FOLDER>/<VM_NAME>.vmx /vmfs/volume/<DATASTORE_NAME>/<VM_HOME_FOLDER>/<VM_NAME>.vmx.back

6. Edit the virtual machine configuration file and remove the entry reading config.readOnly = "TRUE"
   1. Open the configuration file using vi 

      vi /vmfs/volume/<DATASTORE_NAME>/<VM_HOME_FOLDER>/<VM_NAME>.vmx

   2. Move the cursor to the line reading config.readOnly = "TRUE" using the arrow key
   3. Press "D" key twice to delete the line
   4. Save the file by pressing Escape key followed by :wq
7. Reload the virtual machine (Refer knowledgebase article Reloading a vmx file without removing the virtual machine from inventory)
8. Re do the configuration changes noted in the step 1
9. Power on the virtual machine

Please reach out to Broadcom Support if you need any assistance in running this workflow.