Resignaturing VMFS3 volumes from the command line in ESXi/ESX

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• This issue occurs if you recently changed the Host Mode setting on an HDS storage array.
• VMFS3 volumes are seen as snapshot volumes.
• Volumes are disabled or missing unexpectedly.
• Datastores are missing unexpectedly.
• You recently set the SPC-2 flag on the EMC Symmetrix storage array.
• You recently downgraded the Perc 4e/Di firmware and local datastores are now missing.
• The vmkernel.log file (located in /var/log/) contains the entry:
  
  vmhba0:0:0:1 may be snapshot
  
  
• This issue occurs if you recently performed the steps in On ESX Server Systems with Multiple PERC 4 Controllers, LUNs Become Temporarily Inaccessible After Upgrading the PERC 4 Firmware (5966817).

Environment

VMware ESX Server 3.0.x
VMware ESXi 3.5.x Embedded
VMware ESX Server 3.5.x
VMware ESX 4.0.x
VMware ESXi 4.0.x Installable
VMware ESX 4.1.x
VMware ESXi 3.5.x Installable
VMware ESXi 4.1.x Installable

Cause

Resolution

Note: This article is specific to ESXi/ESX 3.x, even though the EnableResignature setting works in ESXi/ESX 4.x. For the supported method to handle snapshot LUN in ESX/ESXi 4.x and 5.x, see vSphere handling of LUNs detected as snapshot LUNs (1011387).

Making this type of change on the storage array results in assigning a new LUN ID (UUID or Serial Number) to all LUNs presented via the modified ports/FAs.

VMFS3 metadata identifies the volumes by several properties which include the LUN number and the LUN ID (UUID or Serial Number). Because the LUNs now have new UUIDs, the resulting mismatch with the metadata leads to LVM identifying the volumes as snapshots.

You must resignature the VMFS3 volumes to make them visible again.

Caution:

• Before performing the steps in this article:
  • Wait for a maintenance window
  • Make sure all virtual machines are backed up properly
• Do not deviate from these steps.

To resignature the VMFS3 volumes from the command line:

1. Power off all virtual machines on all ESX hosts that are stored on the volume.
2. Run this command to re-read the volume on each ESX host:
   
   # vmkfstools -V
   
   If this works, the problem is resolved. If this does not work, proceed to step 3.
   
   
3. Remove all the virtual machines on that volume from the inventory in the VirtualCenter server.
   
   Warning: Do not delete the virtual machines from disk. Remove them from the inventory.
   
   
4. Log in to each host.
5. Run this command to verify which HBAs are providing access to the LUN:
   
   # esxcfg-mpath -l | less
   
   
6. To set the ESX host to resignature all datastores detected as snapshots on the next rescan, run the following command:
7. • For ESX:
     
     # echo "1" > /proc/vmware/config/LVM/EnableResignature
     
     
   • For ESXi:
     
     # esxcfg-advcfg -s 1 /LVM/EnableResignature
     
     
8. To rescan the storage and perform the resignature on volumes detected as a snapshot, run the command:
   
   # vmkfstools -V
   
   
9. To disable the resignature flag, run the following command:
   
   Warning: Do not skip this step.
   
   
   • For ESX:
     
     # echo "0" > /proc/vmware/config/LVM/EnableResignature
     
   • For ESXi:
     
     # esxcfg-advcfg -s 0 /LVM/EnableResignature
     
     
10. To ensure that the volumes stay mounted after resignaturing is turned off, run the command:
    
    # vmkfstools -V
    
    
11. Reinventory the virtual machines.
    1. Open VI Client to the VirtualCenter Server. The virtual machines show as inaccessible.
    2. Right-click the virtual machine and click Remove From Inventory.
    3. Double-click the datastore to bring up the datastore browser.
    4. Select the folder in which the virtual machines resides.
    5. Right-click the virtual machine file (extension .vmx) or template file (extension .vmtx)
    6. Click Add to Inventory.
    7. Follow the wizard.

Additional Information

For related information, see Resignaturing VMFS3 volumes on VMware ESX 3.x via the VMware Infrastructure Client (9453805).

To resignature LUNs on ESX/ESXi 4.0/4.1 for DR purposes:

1. Run this command and verify if the hosts are seeing the volumes as snapshots:
   
   esxcfg-volume -l
   
   
2. If some of the hosts are seeing the LUNs and others are not, shut down and remove the virtual machines using these LUNs from inventory
   
   Note: Ensure not to remove the virtual machines from the disk.
   
   
3. Run this command to enable resignaturing snapshot volumes on rescan:
   
   esxcfg-advcfg -s 1 /LVM/EnableResignature
   
   
4. Run this command and re-read the volumes:
   
   vmkfstools -V
   
   
5. Run this command and verify that the hosts do not see the volumes as snapshots:
   
   esxcfg-volume -l
   
   
6. Run this command to disable resignaturing snapshot volumes on rescan:
   
   esxcfg-advcfg -s 0 /LVM/EnableResignature
   
   
7. In vCenter Server, refresh datastores on each host that had the issue. for the resignatured datastores, you should see a display similar to:
   
   snap-abc123-name
   
   
8. Rename the datastores as required.
9. Browse datastores and add virtual machines to the inventory.

vSphere handling of LUNs detected as snapshot LUNs
On ESX Server Systems with Multiple PERC 4 Controllers, LUNs Become Temporarily Inaccessible After Upgrading the PERC 4 Firmware
Resignaturing VMFS3 volumes on VMware ESX 3.x via the VMware Infrastructure Client