SDDC Manager Config Drift failed at "PrepareEsxiServiceAccountsInputAction: Failed to find SSO credentials"

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

1. Config drift failed with final status "COMPLETE_WITH_FAILURE"
2. Running the command below shows the following errors:

• /apps/migrationcoordinator/bin/migration_coordinator_utility --script_name=SddcManagerConfigDrift | jq

[
  {
    "script_name": "SddcManagerConfigDrift",
    "run_id": "50e432e3-4e6c-41dc-aa83-209f8f1554f3",
    "process_id": "9294",
    "run_type": "EXECUTE",
    "run_parameters": "gAAAAABj9RaF04--KUmWuTOEFAeA6G11VEADtaQPWRvYYyiPjfQrBs87Z-arp-HDgcAqBdd9nWKgp_pkCx1dNWZ0H6U7w5sJC2iKprf_GVJgFiTzV4E7ywWQJAOMDI4bqRubmBChB21Lmcip9qPdEYvihZczhhoRCzjJYzcR4tknF2G-CsP5FwsA83z9OdDj9Sn38wlEdgiDCGdCrzCcFJSAeurZ_dQOF7IyUtOLqrSkb1hR9h77Cg8xRrLBH0pnadClrxHrmvYCMxcrAaWWVGbV8ERaUqnGxlq_vHrTYAE9XDkz-Nn9E4rF9_cApQH78sgl5A6lzdhfDa8pXzKWevAz-4fIjelUDb324gs0Ol6ltHvX6bVpUChGFSrACvD9kJy0rENod4sq",
    "target_resource": null,
    "is_retry": false,
    "start_time": 1677006469.2402456,
    "end_time": null,
    "status": "COMPLETE_WITH_FAILURE",
    "error": {
      "error_code": "CONFIG_DRIFT_ERROR",
      "error_message": "Execution of SddcManagerConfigDrift on domain: f4884c33-8f35-41bc-91e5-b63b6e3d39ed failed. \nPrepareEsxiServiceAccountsInputAction: Failed to find SSO credentials.",
      "remediation": "Please check migration log file for error details. After resolving, retry the step",
      "reference_token": "uXl9NI",
      "original_exception": "None"
    },
    "aliasing_metadata": null
  }
]

•  nohup /apps/migrationcoordinator/bin/sddc_manager_config_drift --post-validate-only &

ESC[K[====--------------------------------------------------------] 7.142857142857142% ... Failed
ESC[KLoadInventoryAndCredentialsData                    Successful
ESC[KPrepareEsxiServiceAccountsInputAction              Failed
ESC[KCreateEsxiUser                                     Pending
ESC[KAssignRoleToEsxiUsers                              Pending
ESC[KAddServiceAccountToLockdownModeExceptionUsersList  Pending
ESC[KPersistEsxiServiceAccountsAction                   Pending
ESC[KPrepareClusterTableDriftInputAction                Pending
ESC[KUpdateClusterMoidAction                            Pending
ESC[KUpdateClusterContextAction                         Pending
ESC[KVxRailUpdateCredentialsForMigrationAction          Pending
ESC[KGetVcRemoteEndpointForDomainAction                 Pending
ESC[KChangeLacpModetoEnhancedAction                     Pending
ESC[KUpdateNsxtManagerInventoryAction                   Pending
ESC[KGetVcRemoteEndpointForDomainAction                 Pending
Traceback (most recent call last):
  File "/apps/migrationcoordinator/packages/migration_coordinator/scripts/configdrift/SddcManagerConfigDrift.py", line 472, in <module>
    action.execute()
  File "/apps/migrationcoordinator/packages/migration_coordinator/engine/decorators/Decorator.py", line 87, in __call__
    raise migration_exception
  File "/apps/migrationcoordinator/packages/migration_coordinator/engine/decorators/Decorator.py", line 74, in __call__
    execute_result = self.func(self.instance_, *args, **kwargs)
  File "/apps/migrationcoordinator/packages/migration_coordinator/scripts/configdrift/SddcManagerConfigDrift.py", line 415, in execute
    [self.name, domain.id, error], None)
migration_coordinator.engine.exception.MigrationException.MigrationException: Migration encountered an Error
MigrationErrorCode.CONFIG_DRIFT_ERROR: Execution of SddcManagerConfigDrift on domain: f4884c33-8f35-41bc-91e5-b63b6e3d39ed failed.
PrepareEsxiServiceAccountsInputAction: Failed to find SSO credentials.
Remediation: Please check migration log file for error details. After resolving, retry the step
Original Exception: None
ReferenceToken: uXl9NI

Environment

Vmware Cloud Foundation 4.5
VMware Cloud Foundation 5.0
Vmware Cloud Foundation 4.5.1

Cause

Disconnected/Not-responding hosts in the respective domain.

Resolution

Validate and ensure all hosts are healthy and in a proper state before applying or retrying the Config Drift bundle. 

Workaround:

1. Take a snapshot from the SDDC Manager.
2. Run the following commands via an SSH to the SDDC Manager.

cp /apps/config-drift/conf/application.properties /apps/config-drift/conf/application.properties.bak
vi /apps/config-drift/conf/application.properties

3. Add the disconnected/not-responding host "skip.esx.svc.acc.config.drift.esxHostnames=<hostname>".
4. Run the Config Drift script for the required workload domain.

 

Additional Information

Impact/Risks:
Unable to apply config drift on the desired workload