IP Customization Failure During RHEL VM Recovery in SRM with Error Code 126 or Error Code 255

Products

• VMware Live Recovery
• VMware Cloud on AWS

Issue/Introduction

Symptoms

• During the recovery of a RHEL (Red Hat Enterprise Linux) virtual machine using Site Recovery Manager, you may encounter the following error message:

“Cannot complete customization, possibly due to a scripting runtime error or invalid script parameters (Error code: 126 or 255). IP settings may have been partially applied.”

• This error typically occurs during the IP customization phase of the recovery process, where SRM attempts to configure the IP settings of the VM.
• "vmware-dr.log", may show an error message similar to the following:

--> 2023-10-15T09:04:08 DEBUG: opening file for writing (/etc/resolv.conf).
--> 2023-10-15T09:04:08 DEBUG: Permission denied writing data to file /etc/resolv.conf.
--> 2023-10-15T09:04:08 INFO: File Attributes: /etc/resolv.conf
--> 2023-10-15T09:04:08 DEBUG: Command: 'lsattr /etc/resolv.conf'

• The critical part of this log indicates a "Permission denied" error when attempting to write to the /etc/resolv.conf file. This is followed by an error code 255 or 126, signaling the failure of IP customization.

--> Exit code: 255
--> ",
-->                returnValue = 255
-->             }
-->             msg = "Cannot complete customization, possibly due to a scripting runtime error or invalid script parameters (Error code: 255). IP settings may have been partially applied."
-->          },
-->          (dr.recovery.fault.IPCustomizationFailure) {
-->             faultCause = (vmodl.MethodFault) null,
-->             faultMessage = <unset>,
-->             result = (dr.recovery.CalloutResult) {
-->                commandLine = "echo "Starting IP customization on Linux ..." >> "$VMware_GuestOp_OutputFile"

 

OR

 

-->       "                  <_type>dr.recovery.fault.IPCustomizationFailure</_type>",
-->       "                  <msg>Cannot complete customization, possibly due to a scripting runtime error or invalid script parameters (Error code: 126). IP settings may have been partially applied.</msg>",
-->       "                  <result>",
-->       "                      <_type>dr.recovery.CalloutResult</_type>",
-->       "                      <commandLine>&quot;echo &quot;Starting IP customization on Linux ...&quot; &gt;&gt; &quot;$VMware_GuestOp_OutputFile&quot;",

 

• Further examination of the /var/log/vmware-imc/toolsDeployPkg.log in the guest OS reveals the following error messages related to the issue:

2825-01-08T14:26:51 ERROR: Fatal error occurred during customization !! Customization halted.
2025-01-08T14:26:51 ERROR: Error: Operation not permitted:Error writing data to File (/etc/resolv.conf). Operation not permitted at /var/run/.vmware-imgcust-d0ox8Vv/scripts/Utils.pm line 656.
2825-01-08T14:28:51 INFO: Return code is 252.
2025-01-00714:26:51 DEBUG: Removing lock file /var/lock/ure/gosc 

• This error indicates that the IP customization failed due to insufficient permissions to modify the /etc/resolv.conf file, which is crucial for network configuration during recovery.

Environment

VMware Site Recovery Manager 8.x
VMware Site Recovery Manager 9.x
Guest OS - RHEL (Red Hat Enterprise Linux) 

Cause

The root cause of the issue is that the /etc/resolv.conf file in the RHEL 8 guest OS has the immutable flag set, which prevents any modifications to the file. This flag is preventing SRM from making the necessary changes to the file during the IP customization process.

You can verify this by running the following command in the guest OS:

# lsattr /etc/resolv.conf

If the immutable flag is set, the output will display an "i" in the attribute column:

----i----------- /etc/resolv.conf

The "immutable" flag in Linux means that the file cannot be modified, even by the root user. This is typically set using the chattr command and is used to protect critical system files from modification.

Resolution

To resolve the issue, follow these steps:

• Remove the Immutable Flag:

  1. To allow modifications to the file, remove the immutable flag using the following command:

   

  # sudo chattr -i /etc/resolv.conf

  2. After running this command, verify that the flag has been removed by checking the attributes again:

   

  # lsattr /etc/resolv.conf

  3. The output should now look like this, with no "i" character:

   

  # lsattr /etc/resolv.conf
  ---------------------- /etc/resolv.conf

• Retry the Recovery Process

Additional Information