How does vSphere Guest OS Customization work with cloud-init to customize a Linux VM

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information on how does vSphere Guest OS Customization work with cloud-init to customize a Linux virtual machine. Particularly after a Linux virtual machine is customized and powered-on, Guest OS Customization Specification and/or cloud-init data should be applied to the Linux virtual machine correctly.

Resolution

With evolution of both vSphere Guest OS Customization and cloud-init, there are more than one solution to use them to customize a Linux virtual machine. 
vSphere Guest OS Customization can base on two different engines, one is Perl based customization, the other is cloud-init based customization, see How to switch vSphere Guest OS Customization engine for Linux virtual machine on how to switch between these two engines, and vSphere Guest OS Customization has also added native support for using cloud-init since vSphere version 7.0 Update 3.
cloud-init supports different transports on VMware platform, this means that there are multiple ways to provide data to cloud-init, vSphere Guest OS Customization(cloud-init based customization and native cloud-init customization) is one of these ways, the other common ways are: seed ISO, Guestinfo OVF and Guestinfo Keys.

Solution 1:
Using Perl based customization and cloud-init together.
Prerequisites:
•    Ensure that Perl based customization is used, see How to switch vSphere Guest OS Customization engine for Linux virtual machine .
•    Ensure that VMware Tools version 12.1.5 or later is installed.
•    Ensure that only user data is provided to cloud-init by other transports.     
                                
Linux virtual machine is usually rebooted at the end of the Perl based customization. If cloud-init is still running by the time of rebooting happens, it will be terminated and fail to apply user data to virtual machine. To avoid such scenario, from VMware Tools version 12.1.5, the Perl based customization will not reboot virtual machine until it detects cloud-init execution has finished or fails to finish before timeout, this is to make sure cloud-init can load and apply user data completely without interrupted by rebooting triggered by Perl based customization.

The default timeout is 30 seconds, it can be modified in virtual machine, the valid value is a number from 0 to 1800.

For example:
To set the timeout to 60 seconds, execute below command in virtual machine
vmware-toolbox-cmd config set deployPkg wait-cloudinit-timeout 60
To get the timeout value, execute below command in virtual machine
vmware-toolbox-cmd config get deployPkg wait-cloudinit-timeout

Solution 2:
Using cloud-init based customization

Prerequisites:

•    Ensure that cloud-init based customization is used, see KB How to switch vSphere Guest OS Customization engine for Linux virtual machine.
•    Ensure that VMware Tools version 10.3.5 or later is installed.
•    Ensure that cloud-init version 18.4 or later is installed.
•    Ensure that Datasource OVF (cloud-init ver < 23.1) or Datasource VMware (cloud-init ver >= 23.1) is included in cloud-init Datasource search list.
•    Ensure that the virtual machine is a clean cloud-init instance by running "sudo cloud-init clean --seed".
 

The cloud-init based customization converts Guest OS Customization Specification to metadata and provides it to cloud-init. If there was data provided by other transports at the same time, cloud-init will decide which transport shall be used so that only the data from that transport will be loaded. Please remove data from any other transports if metadata from Guest OS Customization Specification is not applied to virtual machine correctly.

Solution 3:
Using native cloud-init customization
Prerequisites:
•    Ensure that vSphere version is 7.0 Update 3 or later.
•    Ensure that VMware Tools version 11.3.0 or later is installed.
•    Ensure that cloud-init version 21.1 or later is installed.
•    Ensure that Datasource OVF (cloud-init ver < 23.1) or Datasource VMware (cloud-init ver >= 23.1) is included in cloud-init Datasource search list.
•    Ensure that the virtual machine is a clean cloud-init instance by running "sudo cloud-init clean --seed".

The native cloud-init customization is available since vSphere version 7.0 Update 3, both metadata and user data can be added to Guest OS Customization Specification and provided to cloud-init. If there was data provided by other transports at the same time, cloud-init will decide which transport shall be used so that only the data from that transport will be loaded. Please remove data from any other transports if metadata and user data from Guest OS Customization Specification are not applied to virtual machine correctly.

Please find the detailed information of this solution in below links:

• How to customize virtual machine using Cloud-init metadata and user data in vSphere 7.0 Update 3
• vSphere Web Services SDK Programming Guide: Guest Customization Using cloud-init
• vSphere product document: Create and Manage Customization Specifications for Windows and Linux