Upgrading or Patching vCenter Server to 8.0 U2 fails with precheck error "Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System"
search cancel

Upgrading or Patching vCenter Server to 8.0 U2 fails with precheck error "Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System"

book

Article ID: 344775

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • Upgrade pre-check on 8.0 U2 fails with below error message :
---------------------------
Pre-upgrade check result

Error:

Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System!

Resolution:
Verify that the ESX Agent Manager extension is running properly on the source vCenter Server instance and https://VC_IP/eam/mob presents correct data. If log in to the MOB is not successful, try resolving the issue with Upgrading or Patching vCenter Server to 8.0 U2 fails with precheck error "Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System" 

-------------------------
  • For migration from Windows vCenter Server, log file "C:\Users\%username%\AppData\Local\VMware\Migration-Assistant\logs.zip - CollectRequirements_com.vmware.eam_<DATE>.log" will show below error snippets :
    [YYYY-MM-DDTHH:MM:SS]Z INFO eam.lib.eam-upgrade-prechecks Creating local EAM client(port=15005)
    [YYYY-MM-DDTHH:MM:SS]Z ERROR eam Failed to execute trusted certificates check..
    Traceback (most recent call last):
      File "C:\migF629.tmp\PFiles\VMware\CIS\cis_upgrade_runner\payload\component-scripts\eam\lib\eam_ops.py", line 192, in retrieveUrls
    eam.lib.pyEam.VmomiSupport.eam.fault.EamServiceNotInitialized: (eam.fault.EamServiceNotInitialized) {
       msg = 'EAM is still loading from database. Please try again later.',
    }
  • For upgrade from VCSA, log file /var/log/vmware/upgrade/CollectRequirements_com.vmware.eam_<DATE>.log will show below error snippets:  
[YYYY-MM-DDTHH:MM:SS]Z INFO eam.lib.eam-upgrade-prechecks Creating local EAM client(port=15005)
[YYYY-MM-DDTHH:MM:SS]1Z ERROR eam Failed to execute trusted certificates check..
Traceback (most recent call last):
  File "/tmp/vmware-upgrade-temp-dirOfZ3SyDImn/tmplnwe4R2iOv/payload/component-scripts/eam/lib/eam_ops.py", line 192, in retrieveUrls
    allAgencies = esxAgentMgr.agency
eam.lib.pyEam.VmomiSupport.eam.fault.EamServiceNotInitialized: (eam.fault.EamServiceNotInitialized) {
   msg = 'EAM is still loading from database. Please try again later.',
}
  • Patching VCSA from 8.x to 8.0 Update 2 will show below error message :
 
  • vCenter Server patching using CLI shows below errors
Command> software-packages install --staged

[YYYY-MM-DDTHH:MM:SS] : update is already staged. Proceeding to install.

[YYYY-MM-DDTHH:MM:SS] : Installing version: 8.0.2.00000
[YYYY-MM-DDTHH:MM:SS] : Running precheck ....
[YYYY-MM-DDTHH:MM:SS] : Installation failed. Retry to resume from the current state. Or please collect the VC support bundle.
Mismatch:
summary: Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System!
resolution: Verify that the ESX Agent Manager extension is running properly on the source vCenter Server instance and https://VC_IP/eam/mob presents correct data. If log in to the MOB is not successful, try resolving the issue with https://knowledge.broadcom.com/external/article?legacyId=94934.
  • If source VC is 6.x, eam log (/var/log/vmware/eam/eam.log or %ProgramData%/VMware/vCenterServer/logs/eam/eam.log) will show below entries:
[YYYY-MM-DDTHH:MM:SS]Z |  INFO | vim-monitor | VcConnection.java | 199 | Connecting to vCenter as com.vmware.vim.eam extension
[YYYY-MM-DDTHH:MM:SS]Z |  INFO | vim-monitor | VcConnection.java | 640 | Connecting to https://VC_IP/:8089/sdk/vimService via vCenter proxy http://localhost:80
[YYYY-MM-DDTHH:MM:SS]Z | ERROR | vim-monitor | VcConnection.java | 213 | Failed to login to vCenter as extension. vCenter has probably not loaded the EAM extension.xml yet.: Cannot complete login due to an incorrect user name or password.
[YYYY-MM-DDTHH:MM:SS]Z |  WARN | vim-monitor | VcListener.java | 111 | Trying to recover from error
(vim.fault.InvalidLogin) {
   faultCause = null,
   faultMessage = null
}
  • If source VC is 7.x or higher, eam log (/var/log/vmware/eam/eam.log) will show below entries:
[YYYY-MM-DDTHH:MM:SS]Z |  INFO | vim-monitor | OpId.java | 37 | [vim:loginExtensionByCertificate:a8787fc44d91dff8] created from [Retry:Login:com.vmware.vim.eam:64b8d949da327945]
[YYYY-MM-DDTHH:MM:SS]Z |  INFO | vim-async-2 | OpIdLogger.java | 43 | [vim:loginExtensionByCertificate:a8787fc44d91dff8] Failed.
[YYYY-MM-DDTHH:MM:SS]Z |  WARN | vim-async-2 | ExtensionSessionRenewer.java | 227 | [Retry:Login:com.vmware.vim.eam:64b8d949da327945] Re-login failed, due to: com.vmware.eam.security.NotAuthenticated: Failed to authenticate extension com.vmware.vim.eam to vCenter.


Environment

VMware vCenter Server 8.0.x

Cause

As part of the EAM upgrade pre-checks, an EAM client is created to retrieve all EAM agencies and perform necessary SSL trust checks. This step might not succeed if an EAM client cannot be created because the EAM service is unable to log in to vCenter. This can occur due to a discrepancy between the "vpxd-extension" certificate stored in VECS and the certificate information stored in the vCenter Server Database for the EAM extension.

Resolution

Update the certificate for the Extensions in VPXD by following any of below options: 

Option 1 - Update extensions using fixcerts script from Upgrading or Patching vCenter Server to 8.0 U2 fails with precheck error "Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted certificates by the System"

Update the extension's certificate using fixcerts script:

  • Download the attached fixcerts script
  • Copy the downloaded script to VCSA
  • Execute the script using below arguments to update the extensions
python fixcerts.py update --ExtensionType all

Sample screenshot:

 
Note:

Attachments

fixcerts.py get_app