If deleting a clone of a virtual machine enabled Fault Tolerance previously, unable to disable Fault Tolerance on the original virtual machine.
search cancel

If deleting a clone of a virtual machine enabled Fault Tolerance previously, unable to disable Fault Tolerance on the original virtual machine.

book

Article ID: 433395

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Disabling Fault Tolerance (FT) may failed with error "Disconnected from virtual machine. Remote connection failure Failed to establish transport connection.".

If disabling FT fails, the FT primary VM is powered off and may be restarted by vSphere HA.

You can see simiar the following logs in vmkernel.log at ESXi whichi running FT primary virtual machine.

YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu0:1042696)WARNING: FTCpt: 9490: (4321607309907569883 pri) FT primary host cannot write checkpoint data to secondary host.
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu2:1042697)WARNING: FTCpt: 10484: (4321607309907569883 pri) FT primary host cannot hear from secondary host.
YYYY-MM-DDThh:mm:ss.fffZ In(182) vmkernel: cpu0:1042696)FTCpt: 9496: (4321607309907569883 pri) Checkpoint writer thread exiting
YYYY-MM-DDThh:mm:ss.fffZ In(182) vmkernel: cpu2:1042697)FTCpt: 10488: (4321607309907569883 pri) Cpt Reader exiting
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu4:1042688)WARNING: FTCpt: 2167: (4321607309907569883 pri) Error starting checkpoint: Already disconnected
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu0:1042688)WARNING: FTCpt: 1667: Failed to rename, try lookup generation file: /vmfs/volumes/<Datastore>/<VM>/.ft-generation
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu0:1042688)WARNING: FTCpt: 1671: Expected not found /vmfs/volumes/<Datastore>/<VM>/.ft-generation, actual: Not found
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu4:1042688)WARNING: FTCpt: 1683: (4321607309907569883 pri) Error renaming .ft-generation1 -> .ft-generation2: Not found
YYYY-MM-DDThh:mm:ss.fffZ Wa(180) vmkwarning: cpu4:1042688)WARNING: FTCpt: 4358: (4321607309907569883 pri) Couldn't change generation number: Not found

Environment

VMware vCenter Server 8.0.x

Cause

This issue is occurred that there is no .ft-generation# file regarding FT's generation on the virtual machine's directory.

The case where .ft-generation# file is removed from the virtual machine's directory and disabling FT fails occurs when the following steps are performed:

Steps:

  1. Configure FT at a virtual machine vm_origin.
  2. Disable FT at a vm_origin.
  3. Create a clone of virtual machine vm_clone from a vm_origin.
  4. Reconfigure FT at  a vm_origin.
  5. Delete a vm_clone. (.ft-generation# is removed from the vm_origin's directory.)
  6. Disable FT at a vm_origin. (Disabling FT fails because .ft-generation# does not exist in the vm_origin directory.)

The file path for .ft-generation (ft.lockFile = "/vmfs/volumes/<Datastore>/<virtual machine name>/.ft-generation") remains in the vmx file of a virtual machine vm_origin even after FT is disabled at Step2. Therefore, when a virtual machine vm_clone is cloned from a vm_origin at Step3, the ft.lockFile is copied to the vmx of a vm_clone. The ft.lockFile path of a vm_clone is same path of a vm_origin, so deleting the vm_clone at Step5 also deletes the .ft-generation# in ft.lockFile path. As a result, there is no .ft-generation# file on directory of vm_origin.

Resolution

After failing to disable FT configuration, confirm that .ft-generation# has been recreated in the virtual machine's directory. Once you have confirmed that .ft-generation# exists, attempt to disable FT configuration again.

 

Pre-Action to avoid this issue:

To avoid deleting .ft-generation# as follow either of the following action:

  • Update to vCenter Server 8.0U3i and later. In vCenter Server 8.0U3i, the process when disabling FT has been fixed to delete the ft.lockFile from the .vmx file.
  • After disabling FT, follow the workaround below.

         Workaround

    1. Disable FT at a virtual machine.
    2. Connect CLI at ESXi that running virtual machine of Step1 via SSH.
    3. Edit the vmx file using the vi editor as follows:

      vi /vmfs/volumes/<Datastore>/<virtual machine name>/<virtual machine>.vmx

    4. Remove the following entry from the vmx file of virtual machine at Step1.

      ft.lockFile = "/vmfs/volumes/<Datastore>/<virtual machine name>/.ft-generation"

    5. Save vmx file.

Additional Information

Fix information of this issue is provided VMware vCenter 8.0 Update 3i Release Notes under "vSphere High Availability Issues" in "Resolved Issues".

vSphere High Availability Issues

PR 3621023: In very rare cases, during turning off vSphere Fault Tolerance (FT) or an FT failover, the primary VM might fail
In very rare cases, while Fault Tolerance is turning off or an FT failover triggers, the primary VM might fail.

This issue is resolved in this release.



Japanese Version:
過去に Fault Tolerance を有効化した仮想マシンからのクローンを削除するとクローン元の仮想マシンで Fault Tolerance を無効化できない

Powered by Wolken Software