Post vCenter upgrade from any version less than 8.0.2 to 8.0.2, FDM agents are unreachable in vCenter Server UI

Products

• VMware vSphere ESXi

Issue/Introduction

The following details could be used as a pre-check before the vCenter Server upgrade from any version less than 8.0.2

Symptoms:

After the vCenter upgrade from any version less than 8.0.2 to 8.0.2, the FDM agent crashes immediately after they start on many hosts.

This may be due to the Advanced config setting: das.config.vmacore.ssl.sslOptions set in a HA enabled cluster before the upgrade.

 

Below is the sample log snippet to identify the issue:

vobd.log:
YYYY-MM-DD HH:MM:SS: [UserWorldCorrelator] 31075001277491us: [vob.uw.core.dumped] /opt/vmware/fdm/fdm/fdm(19663736) /var/core/fdm-zdump.000
YYYY-MM-DD HH:MM:SS: [UserWorldCorrelator] 31075219211449us: [esx.problem.application.core.dumped] An application (/opt/vmware/fdm/fdm/fdm) running on ESXi host has crashed (1 time(s) so far). A core file may have been created at /var/core/fdm-zdump.000.

fdm.log:
YYYY-MM-DD HH:MM:SS info fdm[19663736] [Originator@6876 sub=Default] Creating SSL Contexts
YYYY-MM-DD HH:MM:SS error fdm[19663736] [Originator@6876 sub=Libs] error [ConfigStore:7a38895800] [2000] Invalid type specified for: ssl_options
YYYY-MM-DD HH:MM:SS info fdm[19663736] [Originator@6876 sub=Libs] info [ConfigStore:7a38895800] ConfigStoreException: [context]zKq7AVECAQAAAEuUVQENY29uZmlnc3RvcmUAgFdBiwFmZG0AgF7PiwGA8teLAQCCQO8AqkfvgHlFZAGACk1kAYDAImQBAESy7QDWztgAnbjYAV0dAmxpYmMuc28uNgAAma/Y[/context]

Cause

HA advanced setting: das.config.vmacore.ssl.sslOptions is configured which is not needed.

Resolution

This issue is resolved in vCenter 8.0.3

Workaround:

Remove the parameter from the Advance settings before the upgrade using the details shared below:

1. From vSphere HA advanced options, remove the configuration :

"das.config.vmacore.ssl.sslOptions=386531328"

2. From VCDB, verify the above configuration is removed :

VCDB=# select DAS_OPTIONS from VPX_COMPUTE_RESOURCE;

3. In ESXi, validate that the following details are not present in "/opt/vmware/fdm/fdm/prettyPrint.sh clusterconfig"

<option>

<key>das.config.vmacore.ssl.sslOptions</key>

<value xsi:type="xsd:string">386531328</value>

</option>

4. The config option settings on the /etc/opt/vmware/fdm/fdm.cfg (in ESXi) WILL NOT be removed even if it has been removed in the file mentioned in the previous step.

Please manually remove this config from fdm.cfg file :

<ssl>

<sslOptions>386531328</sslOptions>

</ssl>

 

Additional Information

Impact/Risks:

HA reconfiguration will fail post vCenter upgrade, which can lead to VMs going down if there is a potential issue with a node.