Troubleshooting guest agents in VMware vRealize Automation

Products

VMware Aria Suite

Issue/Introduction

This article provides steps to troubleshoot Windows guest agent of vRealize Automation

Symptoms:

When vRealize Automation (formerly known as vCloud Automation Center) Windows guest agent fails, you experience these symptoms:

Provisioning Windows machines fail in a CustomizeOS state before getting destroyed after 2 hours by the default timeout.
Provisioning Windows machines fail due to SSL certificate failures logged in the Axis2 logs.
If the incorrect vRealize Automation server addresses are configured during the guest agent configuration, provisioning Windows machines fail.
If the downloaded Guest Agent zip file was not unblocked before unzipping to the guest, provisioning Windows machines fail.

Environment

VMware vRealize Automation 7.x
VMware vRealize Automation 6.x

Resolution

To troubleshoot the Windows guest agent for vRealize Automation, perform these troubleshooting steps on a newly provisioned machine. If these steps resolve the issue, the guest agent completes its assigned tasks, remove itself from running services, and the machine ends up in an On state within the vRealize Automation.

Ensure (at least for testing) that you are using the custom properties:

VirtualMachine.Admin.UseGuestAgent = true and VirtualMachine.Customize.WaitComplete = true
Ensure that you can ping the FQDN of the vRealize Automation and IaaS Manager Service server. If you cannot, resolve this issue before proceeding to step 3.
For additional testing you may want to try using the delay custom property to verify that everything is complete prior to attempting guest customization:

VirtualMachine.Admin.CustomizeGuestOSDelay = 00:05:00 (for 5 minutes).
Stop the guest agent service from the services in the Windows.
Uninstall guest agent by running this elevated command:

c:\vrmguestagent\winservice -u
Refresh the Windows Services to confirm that the guest agent is no longer present.
Delete the VRMGuestAgent folder.
Download the guest agent to the guest again (if using vRealize Automation 6.x, you can access from https://vcacFQDN/software.
Right-click the downloaded zip file and select properties.
Unblock the zip file.
Extract the contents of the zip file to the root:

c:\VRMGuestAgent\
Reinstall by running this command:

winservice -i -h FQDN IaaS Manager Service server:443 -p ssl
From the Windows Services, start the service.
Verify that the guest agent has removed itself from the Services and machine has passed the CustomizeOS state.

To troubleshoot the Linux guest agent for vRealize Automation, perform these troubleshooting steps on a newly provisioned machine. If these steps resolve the issue, the guest agent completes its assigned tasks, remove itself from running services, and the machine ends up in an On state within the vRealize Automation.

Ensure (at least for testing) that you are using the custom properties:

VirtualMachine.Admin.UseGuestAgent = true and VirtualMachine.Customize.WaitComplete = true
Ensure that you can ping the FQDN of the vRealize Automation and IaaS Manager Service server. If you cannot, resolve this issue before proceeding to step 3.
For additional testing you may want to try using the delay custom property to verify that everything is complete prior to attempting guest customization:

VirtualMachine.Admin.CustomizeGuestOSDelay = 00:05:00 (for 5 minutes).
Uninstall the Linux guest agent by running this command:

rpm –e name_of_package

You can verify the package name by running the command:

rpm –qa |grep –i gugent
For Ubuntu Distributions: "The guest agent files you download contain both tar.gz and RPM package formats. If your operating system cannot install tar.gz or RPM files, use a conversion tool to convert the installation files to your preferred package format."

Note: apt or yum can be leveraged for the removal accordingly based upon Distribution

To remove the Linux guest agent directory, run the command:

rm –rf /usr/share/gugent
Download the appropriate Linux guest agent from https://FQDN_of_vRA/software.
Install the guest agent using this command:

rpm -i gugent-(version).rpm
Change directory to /usr/share/gugent and run the configuration command:

/installgugent.sh FQDN_of_Manager_Service:443 ssl
Install the correct cert.PEM file into the directory by running this command:

echo | openssl s_client -connect FQDN_of_Manager_Service:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > cert.pem
Reboot the Linux server and monitor the /usr/share/gugent/axis2/logs/axis2.log file for connectivity details indicating that the connection succeeded.
Monitor the /usr/share/gugent/guestagent.log file for details on items being retrieved from vRealize Automation.

Note:

If these steps resolve your issue, you need to perform the same steps against the template used to provision this machine so that all future provisioning completes successfully.
If these steps do not resolve your issue, gather full contents of the c:\VRMGuestAgent or /usr/share/gugent/ folder and log a new service request for further assistance. For more information, see Creating and managing Broadcom support cases (142884).

Additional Information

Infrastructure as a Service

See the above link for a more detailed breakdown of each component. See below for a brief breakdown:

Web Server
- Default Web Sites and application pools running in Internet Information Services
Model Manager
- Co-located with the 1st Web Server component
Manager Service
- Can be installed with or on a separate Windows system than the Web Server component
  - Service name within services.msc: "VMware vCloud Automation Center Service"
SQL Server Database
- Hosted on a Microsoft SQL Server instance
Distributed Execution Manager
Agents
- The below is an example only as installed Agents can have differing names: Service name within services.msc: "VMware vCloud Automation Center Agent - vCenter"