VCF 9.x Operations fleet management deployment workflow status error: "Failed to deploy VCF Operations fleet management on SDDC Manager"

Products

• VMware Cloud Foundation
• VMware SDDC Manager

Issue/Introduction

• VCF 9.x deployment fails at Join the existing fleet management appliance stage.
  
  
  
  
• Under VCF 9 deployment subtask, the task fails with Error: "Failed to deploy VCF Operations fleet management on SDDC Manager"
  
  
  
  
• Under SDDC Manager's /var/log/vmware/vcf/domainmanager/domainmanager.log, below log snips are found:
  

   Addition","description":"Prepare Hostnames for VCF Operations fleet management Product SSH Keys Addition","status":"SUCCESSFUL","creationTimestamp":"YYYY-MM-DDTHH:MM:SS","completionTimestamp":"YYYY-MM-DDTHH:MM:SS","errors":[]},{"name":"Add Entries to SDDC Manager Hosts File","description":"Add Entries to SDDC Manager Hosts File","status":"FAILED","creationTimestamp":"YYYY-MM-DDTHH:MM:SS","completionTimestamp":"YYYY-MM-DDTHH:MM:SS","errors":[{"errorCode":"VREALIZE_SSH_KEY_UPDATE_FAILED","errorType":"ERROR","arguments":[],"context":{},"message":"Failed to add the VMware Aria entries to SDDC Manager known hosts file","causes":[],"referenceToken":"TOKEN"}]},{"name":"Configure NTP on VMware Aria Suite Lifecycle","description":"Configure NTP on VMware Aria Suite Lifecycle","status":"PENDING","creationTimestamp":"YYYY-MM-DDTHH:MM:SS","completionTimestamp":"YYYY-MM-DDTHH:MM:SS","errors":[]},{"name":"Automation Helper Action","description":"Automation Helper Action","status":"PENDING","creationTimestamp"

  YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,############,####] [c.v.e.s.s.GetVcfOpsMgmtDeploymentStatusAction,dm-exec-##]  VCF Operations Management deployment workflow failed
YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,############,####] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-##]  [TOKEN] FAILED_TO_DEPLOY_VCF_OPS_MGMT Failed to deploy VCF Operations fleet management on SDDC Manager
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to deploy VCF Operations fleet management on SDDC Manager
        at com.vmware.evo.sddc.sddcmanager.GetVcfOpsMgmtDeploymentStatusAction.execute(GetVcfOpsMgmtDeploymentStatusAction.java:156)
        at com.vmware.evo.sddc.sddcmanager.GetVcfOpsMgmtDeploymentStatusAction.execute(GetVcfOpsMgmtDeploymentStatusAction.java:39)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:66)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:161)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:147)

 

Environment

• VMware Cloud Foundation 9.x
• SDDC Manager 9.x

Cause

This issue can occur due to either of below issues:

• Port 22 and 443 is not opened between SDDC Manager and VCF Operations Manager. 
• SSH is disabled either in SDDC Manager or VCF Operations Manager or on both.

Resolution

To resolve this issue, follow below options as applicable:

Option 1: Open the Port 22 and 443 between SDDC Manager and VCF Operations Manager (bi-directional). 

Option 2: Enable SSH on both SDDC Manager and VCF Operation Manager.