SRM IP Customization "Failed to authenticate with the guest operating system" due to NTP issues
search cancel

SRM IP Customization "Failed to authenticate with the guest operating system" due to NTP issues

book

Article ID: 389190

calendar_today

Updated On:

Products

VMware Live Recovery

Issue/Introduction

  • IP customization fails on Windows/Linux VM. Plan completes with below error message 

         Error - 'Failed to authenticate with the guest operating system using the supplied credentials'

  • SRM server logs report:

/var/log/vmware/srm/vmware-dr.log

-->    msg = "Received SOAP response fault from [<SSL(<io_obj p:0x00007fd9a0003e20, h:47, <TCP '10.x.x.x : 55142'>, <TCP '10.x.x.x  : 443'>>), /sdk>]: createTemporaryDirectory
--> Failed to authenticate with the guest operating system using the supplied credentials."
--> }
2025-02-04T19:49:13.743+06:00 warning vmware-dr[02296] [SRM@6876 sub=Recovery opID=ed092a7f-0d23-4859-9b33-088543c111c6-failover:83e6:fdc3:521d:dc63:ad36:4517]  [4d90b1dc-f38b-4cbc-b8d2-88a9e5bf4d6f.ipCust-protected-vm-XXXX-IPCustomization] CREATE_TEMP_FOLDER_STEP  Retrying (2 of 22) the guest operation after 10 seconds due to: (vim.fault.InvalidGuestLogin) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>
-->    msg = "Received SOAP response fault from [<SSL(<io_obj p:0x00007fd9a0003e20, h:47, <TCP '10.x.x.x  : 55142'>, <TCP '10.x.x.x  : 443'>>), /sdk>]: createTemporaryDirectory
--> Failed to authenticate with the guest operating system using the supplied credentials."
--> }
--> [context]zKq7AVECAAQAANjOcAEKdm13YXJlLWRyAAAsGRxsaWJ2bWFjb3JlLnNvAIF3ixcBbGl

  • Guest OS Logs:

         Guest VGAuth logs:

         Windows OS: \ProgramData\VMWare\VMWare VGAuth\logfile*

         Linux OS: /var/log/vmware-vgauthsvc.log

 

 [   debug] [VGAuthService] Pref_GetInt: Pref_GetInt(clockSkewAdjustment) failed: Key file does not have key “clockSkewAdjustment” in group “service”
 [ message] [VGAuthService] LoadPrefs: Allowing 300 of clock skew for SAML date validation
 [ message] [VGAuthService] SAML_Init: Using xmlsec1 1.2.20 for XML signature support
 [   debug] [VGAuthService] SendRpciPacket: RPC returned 'Unknown command'
 [   debug] [VGAuthService] Pref_GetInt: Pref_GetInt(listenTTL) failed: Key file does not have key “listenTTL” in group “service”
 [   debug] [VGAuthService] ServiceInitListenConnectionPrefs: listen conn TTL set to 1800 seconds
 [   debug] [VGAuthService] ServiceInitListenConnectionPrefs: computed reapCheckTime as 180 seconds
 [   debug] [VGAuthService] Pref_GetInt: Pref_GetInt(maxDataConnectionsPerUser) failed: Key file does not have key “maxDataConnectionsPerUser” in group “service”

 

 

 

 

Environment

VMware Site Recovery Manager 8.x
VMware Site Recovery Manager 9.x

Cause

This issue occurs due to mismatch in time synchronization between SRM and ESXi Host.

NTP server is not configured at the DR ESXi Host.

Resolution

Set the NTP server settings on the ESXI Host-

 

Method 1: Using the vSphere Client (GUI)

Configuring Network Time Protocol (NTP) on an ESXi host using the vSphere Client

Method 2: Using ESXi Shell / SSH (CLI)

  1. Access the ESXi Host via SSH:

    • SSH into your ESXi host using a terminal or SSH client.
    • If SSH is not enabled, you can enable it from the Direct Console User Interface (DCUI) under Troubleshooting Options.

    Run the below command to first check current NTP settings on the ESXi Host 

   [root@localhost:~] esxcli system ntp get
   Enabled: false
   Loglevel: warning
   PID: 0
   Runtime Seconds: 0
   Servers: 10.x.x.x
   Service Providing Kernel Time:
   Time Service Enabled: false
   Time Synchronized: false

       2. Use below command to manually set the NTP server setting on ESXi Host if it is failing from the vSphere Client

                    esxcli system ntp set --server=ntp0.ntp-servers.net

 

    Post which attempt to run the Recovery Plan again, this time the plan should complete along with IP customization.

 

 

 

 

         

 

Additional Information

  • If IP customization hits guest authentication failure, guest OS logs from the recovered VM, not the production VM, need to be collected besides SRM log bundle. Follow below steps to change log level configuration in production VM, and sync latest changes to recovery site, then re-run test failover and collect the logs from recovered VM.

    1. VGAuth service debug logs plus aliases store content.
      VGAuth config file location
      • Windows OS: %programdata%\VMware\VMware VGAuth\vgauth.conf
      • Linux OS: /etc/vmware-tools/vgauth.conf


      The logging level is in the       [service] section. The default value is normal. Changing it to verbose will provide the best debugging information. NOTE: Be sure to restore the log level after logs are collected. Debug logs may contain SAML tokens which could be used to attack other VMs.

      Log level
      [service]
      loglevel = verbose


      VGAuth log file location

      • Windows OS: \ProgramData\VMWare\VMWare VGAuth\logfile*
      • Linux OS: /var/log/vmware-vgauthsvc*

    2. All log files part of vmware-imc folder.
      • Windows OS: C:\windows\temp\vmware-imc\ or %SystemRoot%\Temp\vmware-imc\
      • Linux OS: /var/log/vmware-imc/..

    3. Debug VMTools logs.

      • Browse to the configuration file location as specified per operating system.  If the files does not exist create the file with a text editor

        Guest operating system
        Path to configuration file
        Windows XP and Windows Server 2000/2003 C:\Documents and Settings\All Users\Application Data\VMware\VMware Tools\tools.conf
        Windows Vista, Windows 7, Windows 8, Windows Server 2008 and Windows Server 2012 C:\ProgramData\VMware\VMware Tools\tools.conf
        Linux, Solaris, and FreeBSD /etc/vmware-tools/tools.conf
        Mac OS X /Library/Application Support/VMware Tools/tools.conf
      • Add and save the following lines to the configuration file, depending on the operating system.

         

        Windows
        Linux
        		 
        [logging] 
        log = true 
        vmtoolsd.level = debug 
        vmtoolsd.handler = file 
        vmtoolsd.data = c:/tmp/vmtoolsd.log
        		 
        [logging] 
        log = true 
        vmtoolsd.level = debug 
        vmtoolsd.handler = file 
        vmtoolsd.data = /tmp/vmtoolsd.log
      • Restart the VMware Tools service.
        VMware tools process
        • Windows OS: vmtoolsd.exe in Windows guests, and can be managed from the Services application in Windows
        • Linux OS:  vmtoolsd, and it is run from /usr/sbin/vmtoolsd in Linux guests
Powered by Wolken Software