Verifying and reinstalling the correct version of the VMware vCenter Server agents
search cancel

Verifying and reinstalling the correct version of the VMware vCenter Server agents

book

Article ID: 308505

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article provides steps to determine the version of the vCenter Server agent that is installed on an ESX host, and also provides steps to reinstall the VMware HA and vpxa agents if the version is incorrect.

Symptoms:
  • Configuring VMware High Availability (HA) fails.
  • You see the error:

    Could not Enable aam firewall ruleset :vim.fault.HostConfigFault
  • Cannot connect an ESX host to VirtualCenter / vCenter Server.
  • You see the error:

    unable to access the specified host, either it doesn't exist, the server software is not responding, or there is a network problem
  • In the hostd.log file, you see entries similar to :

    [<YYYY-MM-DD>T<time> 'Vmomi' 15752112 info] Throw vim.fault.AlreadyExists
    [<YYYY-MM-DD>T<time></time></time>    'Vmomi' 15752112 info] Result:
    (vim.fault.AlreadyExists) {
    name = "vpxuser"
    msg = ""
    }
<YYYY-MM-DD>T<time> F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Act
ivation [N5Vmomi10ActivationE:0x5bd7b5d8] : Invoke done [enableRuleset] on [vim.host.FirewallSystem:firewallSystem]</time>
[<YYYY-MM-DD>T<time> F5769B90 verbose 'Vmomi' opID=EE62B2D1-0000184B-ec-ab]Arg id:"aam"
<YYYY-MM-DD>T<time> F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Throw vim.fault.NotFound
[<YYYY-MM-DD>T<time> F5769B90 info 'Vmomi' opID=EE62B2D1-0000184B-ec-ab] Result:(vim.fault.NotFound) { dynamicType = <unset>, faultCause = (vmodl.MethodFault) null, msg = "",}
</time></time></time>

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Environment

VMware ESX Server 3.0.x
VMware ESXi 3.5.x Installable
VMware vCenter Server 5.1.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 6.0
VMware vCenter Server 5.0.x
VMware ESXi 3.5.x Embedded
VMware vCenter Server 6.0.x
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.0
VMware ESXi 4.1.x Installable
VMware ESXi 4.0.x Embedded
VMware ESX 4.0.x
VMware ESX Server 3.5.x
VMware VirtualCenter 2.5.x
VMware vSphere ESXi 5.5
VMware vCenter Server 5.5.x
VMware vCenter Server 4.1.x
VMware VirtualCenter 2.0.x

Resolution

This article includes these sections:

  1. Verifying that the correct version of vCenter Server agents are installed on ESX
  2. Reinstalling the agents on ESX
  3. Reinstalling the VMware HA components on ESX
  4. Verifying that the correct version of vCenter Server agents are installed on ESXi
  5. Reinstalling the ESX host management agents and HA agents on ESXi
  6. Manually reinstalling the agents on ESX
  7. Manually installing the VirtualCenter Server agent

Note: Proceed directly to Reinstalling the agents on ESX if you see this error when you try to connect an ESX host to vCenter Server:

Unable to access the specified host, either it doesn't exist, the server software is not responding, or there is a network problem.

Verifying that the correct version of vCenter Server agents are installed on ESX

To verify that the correct version of vCenter Server is installed:

  1. Determine the version and build number of your vCenter Server installation.
    Click Help > About from within vCenter Server.
  2. Right-click the ESX host and click Disconnect to disconnect the ESX host from vCenter Server.
  3. Log in as root to the ESX host using an SSH client.
  4. Run this command to determine the version of the vCenter Server agent (vpxa ) that is installed:

    [root@esx-server /]# rpm -qa | grep vpxa

    You see output similar to:

    VMware-vpxa-2.5.0-64192

  5. Ensure that the output matches the information from step 1. If the versions do not match, reinstall the agents.

Reinstalling the agents on ESX

To reinstall the agents on ESX:

  1. Disconnect the ESX host in vCenter Server. Right-click the host and click Disconnect.
  2. Log in as root to the ESX host using an SSH client.
  3. Run this string of commands in the SSH session:

    service mgmt-vmware stop && service vmware-vpxa stop && service vmware-vmkauthd stop && service xinetd restart && rpm -qa | grep -i vpxa | awk '{print $1}' | xargs rpm -ef $1 && userdel vpxuser && rpm -qa | grep -i aam | awk '{print $1}' | xargs rpm -ef $1 && service mgmt-vmware start && service vmware-vmkauthd start

    Note: This is a concatenated list of commands strung together with the "&& " operator. This is useful in stringing together multiple commands as the next command does not start until the previous one has completed successfully.

    These are the commands you are running:
    • service mgmt-vmware stop && service vmware-vpxa stop && service vmware-vmkauthd stop && service xinetd restartstops the management services for the service console.
    • rpm -qa | grep -i vpxa | awk '{print $1}' | xargs rpm -ef $1 removes the vpxa agent.
    • userdel vpxuser removes the vpxa user that is installed once you connect the host to vCenter Server.
    • rpm -qa | grep -i aam | awk '{print $1}' | xargs rpm -ef $1 removes the AAM RPM (HA) which is reinstalled when you configure HA.
    • service mgmt-vmware start && service vmware-vmkauthd start restarts the services.
    • vmware-cmd -l lets you know when hostd is up and running so that you can reattach it to vCenter Server.

    Note: When you see a list of virtual machines output, the command sequence is complete.

  4. Log in to the vCenter Server using VMware Infrastructure/vSphere Client.
  5. Choose the ESX host from the Inventory.
  6. Right-click on the ESX host and click Connect. This step initiates a re-install of the agent.

Reinstalling the VMware HA components on ESX

To reinstall the VMware HA components:

  1. Remove the ESX host from the vCenter Server Inventory.
  2. Log in as root to the ESX service console.
  3. Run this command:

    rpm -qa | grep -i aam.

    This returns two packages that are named similar to:
    • VMware-aam-haa-#.#.#-#
    • VMware-aam-vcint-#.#.#-#

  4. Run these commands to remove the packages returned by Step 3:

    rpm -e VMware-aam-vcint-#.#.#-#
    rpm -e VMware-aam-haa-#.#.#-#

    Note: Ensure that you remove the VMware-aam-vcint-#.#.#-# package first.

  5. Run this command:

    rpm -qa | grep -i vpxa

    A package named VMware-vpxa-#.#.#-##### is returned.

  6. Run this command to remove the package returned by Step 5:

    rpm -e VMware-vpxa-#.#.#-#####

  7. Try adding the host to the newly created cluster to see if this has resolved the issue.

Verifying that the correct version of vCenter Server agents are installed on ESXi 3.5 and ESXi 4.x

Note: vpxa cannot be uninstalled on an ESXi 5.0 host as it is now a part of the esxi-base package. Therefore, verifying the version and the uninstall/reinstall process does not apply to ESXi 5.0.
 
To verify that the correct version of vCenter Server agents are installed:
  1. Determine the version and build number of your vCenter Server installation.
    Click Help > About from within vCenter Server.
  2. Right-click the ESXi host and click Disconnect to disconnect the ESXi host from vCenter Server.
  3. Log in to ESXi Shell. For more information, see Using ESXi Shell in ESXi (311213).
  4. Run this command to determine the version of the vCenter Server agent (vpxa ) that is installed:

    For ESXi 3.5 and ESXi 4.x:

    [root@esx-server /]# cat `ls -t /var/log/vmware/vpx/vpxa*` | head -1

    You see output similar to:

    Log for VMware VirtualCenter Agent, pid=1976, version=2.5.0, build=build-147633, option=Release, section=394
  5. Ensure that the output matches the information from step 1. If the versions do not match, reinstall the agents.

Reinstalling the ESX host management agents and HA agents on ESXi

To reinstall the ESX host management agents and HA agents:

  1. Right-click the ESXi host and click Disconnect. This disconnects the ESXi host from the vCenter Server Inventory.
  2. Log in to ESXi Troubleshooting Mode. For more information, see Using ESXi Shell in ESXi (311213).
  3. Uninstall the vpxa and aam components from the VMware ESXi host by running these commands:

    For ESXi 3.5 and ESXi 4.x:

    /opt/vmware/uninstallers/VMware-vpxa-uninstall.sh
    /opt/vmware/uninstallers/VMware-aam-ha-uninstall.sh

    Note: Run the cat /etc/shadow command to verify if the vpxuser is removed by the uninstall script. If the vpxuser is not removed, run userdel vpxuser.

    For ESXi 5.0:

    Note: This command uninstalls the HA agent (FDM) on ESXi 5.0 as aam functionality is no longer used for HA on vCenter Server 5.0. For more information on HA (Fault Domain Manager) troubleshooting steps, see Troubleshooting VMware High Availability (HA) issues in VMware vCenter Server 5.x and 6.0 (2004429).

    cp /opt/vmware/uninstallers/VMware-fdm-uninstall.sh /tmp
    chmod +x /tmp/VMware-fdm-uninstall.sh
    /tmp/VMware-fdm-uninstall.sh
  4. Right-click the ESXi 3.5 /ESXi 4.x host and click Connect. This initiates a re-install of the vpxa agent.

    Note: In ESXi 5.0, cannot re-install vpxa as it is now a part of esxi-base package.

  5. Add the host to the appropriate cluster. This initiates a re-install of the aam agent. If the agent fails to automatically install when connecting the ESX host in the vCenter Server:

    1. Verify that /tmp/vmware-root exists on the ESX host. vCenter stages the vpxa RPM in this folder. To add this directory, run the command:

      mkdir -p /tmp/vmware-root

    2. For more information on investigating disk space problems on ESX hosts, see Investigating disk space on an ESX or ESXi host (1003564).

If you experience problems with this method, manually re-install the agents.

Manually reinstalling the agents on ESX

To manually reinstall the agents:

  1. Disconnect the ESX host in VirtualCenter. Right-click the Server and click Disconnect.
  2. Log in as root to the ESX host using an SSH client.
  3. Run these commands to get the names of the packages that need to be removed:

    [root@esx-server /]# rpm -qa | grep vpxa
    VMware-vpxa-2.5.0-64192

    [root@esx-server /]# rpm -qa | grep aam
    LGTOaama-5.1.2-1
    LGTOaamvm-5.1.2-1

    Note: The *aam * packages might not be installed.

  4. Run this command for each of the packages:

    [root@esx-server /]# rpm -e PackageName

    Where PackageName is the name of the package that you are removing.

    Note: If you remove the aam agent from a host, you need to also remove vpxa .

  5. Reconnect the ESX host in VirtualCenter. VirtualCenter automatically installs the required packages.

Manually installing the VirtualCenter Server agent

If the installation fails, it may be necessary to manually install the agent.

To manually install the agent:

  1. Disconnect the ESX host in VirtualCenter. Right-click the host and click Disconnect.
  2. Log in as root to the ESX host using an SSH client.
  3. Run these commands to get the names of the packages that need to be removed:

    [root@esx-server /]# rpm -qa | grep vpxa
    VMware-vpxa-2.5.0-64192

    [root@esx-server /]# rpm -qa | grep aam
    LGTOaama-5.1.2-1
    LGTOaamvm-5.1.2-1

    Note: The LGTOaam * packages might not be installed.

  4. Run this command for each package:

    [root@esx-server /]# rpm -e PackageName

    Where PackageName is the name of the package that you are removing.

  5. Log in to the vCenter Server as Administrator.
  6. Open the folder for the VirtualCenter/vCenter Server installation.
    • For VirtualCenter 2.0.x, the default folder is C:\Program Files\VMware\VMware VirtualCenter 2.0\upgrade
    • For VirtualCenter 2.5.x, the default folder is C:\Program Files\VMware\Infrastructure\VirtualCenter Server\upgrade
    • For vCenter Server 4.x, the default folder is C:\Program Files\VMware\Infrastructure\VirtualCenter Server\upgrade
    • For vCenter Server 5.x, the default folder is C:\Program Files\VMware\Infrastructure\VirtualCenter Server\upgrade
    • For vCenter Server 6.0, the default folder is C:\Program Files\VMware\vCenter Server\vpxd\upgrade
    • For vCenter Server Appliance 5.x, the default folder is /usr/lib/vmware-vpx/upgrade
  7. Open the upgrade sub-folder and look for bundleversion.xml file. This file lists the bundle name for the ESX server version installed on the ESX host. For example, if you are adding an ESX Server 2.5.2 host, you must use bundle 4. The bundle file is a self-extracting shell script.
  8. Using the bundle number, locate this file:

    vpx-upgrade-esx-N-linux-version

    Where N is the bundle number and version is the build number for VirtualCenter.

  9. Use a secure copy utility to copy this file to the ESX host.

    Note: You do not need to copy the .sig file.

  10. Log in as root to the ESX host using an SSH client.
  11. Make the script executable by running this command:

    [root@esxhost root]# chmod +x vpx-upgrade-esx-N-linux-32042

  12. Run this script to install the appropriate RPM and start the vpxa daemons on the ESX host:

    [root@esxhost root]# sh ./vpx-upgrade-esx-N-linux-32042

  13. Log in to the vCenter Server using the VI Client or the vSphere Client.
  14. Connect to the ESX host by selecting it from the Inventory. Right-click the host and click Connect.



Additional Information

Powered by Wolken Software