Storage vMotion migration fails with the error: The method is disabled by 'SYMC-INCR dd-mm-yyyy hh:mm'

Products

• VMware vCenter Server

Issue/Introduction

• Storage vMotion migration fails.
• In vCenter Server 5.0, the Migrate option is unavailable.
• Cannot drag and drop the virtual machine from one folder to another.
• Cannot move virtual machine between folders.
• The virtual machine Remove from Inventory option is unavailable.
• In vSphere Client, you see these errors:
   
  • The method is disabled by '_DP-VEPA_BACKUP_RESTORE_LOCK_'
Error Stack
Call "VirtualMachine.Relocate" for object "VM-NAME" on vCenter Server "vCenter-Name" failed
 
  • The method is disabled by 'SYMC-INCR dd-mm-yyyy hh:mm'
Error Stack
Call "VirtualMachine.Relocate" for object "VM-NAME" on vCenter Server "vCenter-Name" failed
 
  • The method is disabled by 'Data Recovery'
 
  • The method is disabled by 'vSphere Data Protection'
 
  • This method is disabled by 'moref=vm-ID'
 
  • Error:fault.MethodDisabled.summary
     
• Storage vMotion fails with the generic error message:
  
  The method is disabled by XXXXXXXXXXX
   
• When vMotion starts, this message appears:
  
  Invalid / explicitly disabled state operation is invoked

Cause

This issue occurs because the entries from the VPX_DISABLED_METHODS table are not removed after the virtual machine backup task completes. When a virtual machine level backup begins, the backup system informs vCenter Server to disable Storage vMotion for that virtual machine to ensure that the backups can complete successfully. If no backups are active for that virtual machine when this error occurs, Storage vMotion does not automatically re-enable.

Resolution

Note: Please work with 3rd party products Backup vendor to fix their code to cleanly handle enable/disable methods API.

Caution: This procedure modifies the database. Ensure to take a full backup of the existing database before proceeding with any of these options. Involve a Database Administrator if there is any difficulty with these steps.

To work around this issue, use one of these options:

• Schedule another backup
• Manually remove entries from the vCenter Server database
• Manually remove entries from the vCenter Server Appliance vPostgres database
• Remove and re-add the virtual machine from the inventory
• Remove and re-add the ESXi/ESX hosting the virtual machine from the inventory

Schedule another backup

Schedule another backup of the affected virtual machine. If this completes successfully, the entries should be removed from the vCenter Server database and this allows a Storage vMotion of the virtual machine.

Manually remove entries from the vCenter Server SQL database

To manually remove the entries from the vCenter Server database:
 

Caution: This procedure modifies the database. Ensure to take a full backup of the existing database before proceeding with this workaround. Involve a Database Administrator if there is any difficulty with these steps.

1. Stop the vCenter Server services.
2. Take a backup of the vCenter Server database.
3. Run thisSQL statement against the vCenter Server database:
   
   select * from VPX_VM WHERE FILE_NAME LIKE '%Virtual-Machine-Name%'
   
   where Virtual-Machine-Name is the name of the virtual machine. Leave the percentage (%) signs flanking the name.
    
4. Make a note of the vm-ID_Number value returned by the SQL statement in step 3, as this will be required in the SQL statement in steps 5 and 6.
5. To confirm the stale entry, run this SQL statement against the vCenter Server database:
   
   select * from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4'
    
6. To remove the stale entry, run this SQL statement against the vCenter Server database:
   
   delete from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4'
    
7. Start the vCenter Server services, then retry the Storage vMotion. For more information, see Stopping, starting, or restarting vCenter Server Appliance services (315393).

Manually remove entries from the vCenter Server and vCenter Server Appliance vPostgres database

Caution: This procedure modifies the database. Ensure to take a snapshot and/or full backup of the vCenter before proceeding with this workaround.
 

To manually remove the entries from the vCenter Server Appliance vPostgres database for vCenter Server Appliance 5.0 Update 1, 5.1, 5.5, 6.0, and 6.5:
 

1. Connect to the vCenter Server Appliance using SSH. For more information, see Enable or Disable SSH Administrator Login on the VMware vCenter Server Appliance section in the vCenter Server and Host Management Guide.
2. Stop the VMware VirtualCenter Server service using this command:
   
   service vmware-vpxd stop
    
3. Ensure a full backup of the vCenter has been completed.
4. Run this command against the vCenter Server database.
   
   sudo /opt/vmware/vpostgres/1.0/bin/psql -d VCDB vc -c "select * from VPX_VM WHERE FILE_NAME LIKE '%Virtual-Machine-Name%';"
   
   where Virtual-Machine-Name is the name of the virtual machine. Leave the percentage (%) signs flanking the name.
    
5. Make a note of the numerical ID value returned by the SQL statement in step 4, as this will be required in the SQL statement in steps 6 and 7.
6. To confirm the stale entry, run this command against the vCenter Server database:
   
   sudo /opt/vmware/vpostgres/1.0/bin/psql -d VCDB vc -c "select * from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4';"
    
7. To remove the stale entry, run this command against the vCenter Server database:
   
   sudo /opt/vmware/vpostgres/1.0/bin/psql -d VCDB vc -c "delete from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4';"
    
8. Start the VMware VirtualCenter Server service, then retry the Storage vMotion. For more information, see Stopping, starting, or restarting vCenter Server Appliance services (315393).

Windows installed vCenter Server with Postgres database

1. Stop the vCenter Server services.
2. Connect to the Postgres database. For more information, see Connecting to the embedded vPostgres Database in a Windows installed vCenter Server 6.0 (340937).
3. Run this command against the vCenter Server database.
   
   select * from VPX_VM WHERE FILE_NAME LIKE '%Virtual-Machine-Name%';
   
   where Virtual-Machine-Name is the name of the virtual machine. Leave the percentage (%) signs flanking the name.
    
4. Make a note of the numerical ID value returned by the SQL statement in step 4, as this will be required in the SQL statement in steps 6 and 7.
5. To confirm the stale entry, run this command against the vCenter Server database:
   
   select * from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4';
    
6. To remove the stale entry, run this command against the vCenter Server database:
   
   delete from VPX_DISABLED_METHODS WHERE ENTITY_MO_ID_VAL = 'vm-ID from step 4';
    
7. Start the VMware VirtualCenter Server service, then retry the Storage vMotion. F

Remove and re-add the virtual machine from the inventory

To remove and re-add the virtual machine from the inventory:

Caution: Removing a virtual machine from inventory removes any performance statistics and historical data associated with the virtual machine. This process should only be used if the preceding procedures are unsuccessful or if any historical or performance data for a defined virtual machine.

 

1. Power off the virtual machine.
2. In vSphere Client, right-click the virtual machine and click Remove from Inventory.
3. Browse to the datastore where the virtual machine resides.
4. Right-click the .vmx file of the virtual machine and click Add to Inventory.
   
   Note: This gives the virtual machine a new ID.
    
5. Power on the virtual machine and retry the Storage vMotion.

Remove and re-add the ESX/ESXi from the inventory

Caution: Removing an ESX/ESXi host from inventory removes any performance statistics and historical data associated with the host. This process should only be used if the preceding procedures are unsuccessful or if no historical or performance data for the defined ESX/ESXi host needs to be retained.

1. In the vSphere Client, right-click the ESXi/ESX host and click Disconnect.
2. Right-click the disconnected ESXi/ESX host and click Remove from Inventory.
3. After the host is removed from inventory, right-click the cluster and click Add Host.
4. Provide the IP address or hostname of the ESXi/ESX host and the root username and password.

Additional Information

• When Storage DRS is enabled for a datastore cluster, it might be disabled on some virtual disks in the datastore cluster. This may result in the errors similar to those under symptoms section. For more information, see the Storage DRS is Disabled on a Virtual Disk section in the vSphere Troubleshooting Guide.

Enabling a virtual NIC for a virtual machine in a vDS portgroup after a storage migration fails with the error: Invalid configuration for device '0'