Service Account for vCenter Server showing Disconnected in SDDC Manager
search cancel

Service Account for vCenter Server showing Disconnected in SDDC Manager

book

Article ID: 381321

calendar_today

Updated On:

Products

VMware SDDC Manager VMware vCenter Server VMware Cloud Foundation

Issue/Introduction

  • A vCenter Single Sign-On (SSO) service account (for example, svc-<fqdn_of_sddc>@vsphere.local) displays a Disconnected status within the SDDC Manager UI.

  • When attempting to resolve the issue using the Remediate or Rotate password functions in SDDC Manager, the tasks show as successful, but the account status remains Disconnected.

  • Reviewing the Operations Manager log (/var/log/vmware/vcf/operationsmanager/operationsmanager.log) on the SDDC Manager Appliance using SSH Client and logging in as "vcf" and switching to "root" reveals SQL errors indicating a timestamp value is too large:

    YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [c.v.v.p.v.u.CredentialsValidationTaskExecutor,om-exec-2] Updating expiration details for credentialId: fc56bea1-5c2a-4665-####-3f98####6175 in credential expiry cache
    YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [c.v.v.p.s.PasswordExpirationService,om-exec-2] Validation checks size = 1
    YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [c.v.v.p.s.PasswordExpirationService,om-exec-2] expiry cache entity is available : true
    YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [c.v.v.p.s.PasswordExpirationService,om-exec-2] Expiry retrieval status : SUCCEEDED ,  Diagnostic message : null
    YYYY-MM-DDTHH:MM:SS+0000 WARN  [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [o.h.e.jdbc.spi.SqlExceptionHelper,om-exec-2] SQL Error: 0, SQLState: 22008
    YYYY-MM-DDTHH:MM:SS+0000 ERROR [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [o.h.e.jdbc.spi.SqlExceptionHelper,om-exec-2] ERROR: timestamp out of range: "2739933-02-15 01:00:23.326876+00"
      Where: unnamed portal parameter $4 = '...'
    YYYY-MM-DDTHH:MM:SS+0000 ERROR [vcf_om,698d06aa93f1a######85097eeb00f7b,f384] [c.v.v.p.s.PasswordExpirationService,om-exec-2] could not execute statement [ERROR: timestamp out of range: "2739933-MM-DD HH:MM:SS+00"

  • Additionally, querying the passwordmanager.credential_expiry table in the SDDC Manager PostgreSQL database shows that the fetch_time for the affected credential is stuck on an older date and is no longer updating.

  • Command to check the  passwordmanager.credential_expiry table in the SDDC Manager is mentioned here: psql -h localhost -U postgres -d operationsmanager -c "SELECT credential_id, resource_fqdn, username, expiry_date, fetch_time, fetch_status FROM passwordmanager.credential_expiry WHERE resource_type = 'VCENTER' ORDER BY fetch_time DESC;"

Environment

VMware Cloud Foundation 9.x
VMware Cloud Foundation 5.x

Cause

  • This issue occurs when the vCenter Server SSO Password Policy is configured with an expiration value that calculates to an extremely far-future date (such as 0 for "never expires,"  or a very large number like 999999, which vCenter may translate to a year like 2739933).

  • When SDDC Manager attempts to fetch this password expiry date to update its internal PostgreSQL database, the value exceeds the database's maximum supported timestamp limit. This throws a timestamp out of range exception, causing the backend sync to fail and leaving the account permanently in a Disconnected state, regardless of password remediation success.

  • SDDC manager database stores and processes dates in the format YYYY-MM-DD. 

  • Any expiration date after 9999-12-31 will cause the password expiration check to fail and the component will show as disconnected. 

Resolution

To resolve this issue, lower the vCenter SSO password expiration policy to a supported, finite value, and then remediate the password in SDDC Manager.

Step 1: Update vCenter SSO Password Policy

  1. Log in to the vSphere Client (vCenter Server) as an administrator (e.g., [email protected]).

  2. From the main menu, navigate to Administration.

  3. Under Single Sign On, click on Configuration.

  4. Select the Local Accounts tab.

  5. Highlight the Password Policy row and click Edit.

  6. Change the Maximum lifetime (Expiry) to 999 or 9999 days.

  7. Click Save or OK.

Step 2: Retrieve the service accounts credentials from SDDC Manager for the Disconnected Account following KB: Retrieve the service accounts credentials from SDDC Manager

Step 3: Remediate Password in SDDC Manager

  1. Log in to the SDDC Manager UI.

  2. Navigate to Administration > Security > Password Management (or the relevant password management section for your VCF version).

  3. Select the affected service account (e.g., svc-<fqdn_of_sddc>@vsphere.local) that is showing as Disconnected.

  4. Click Remediate.

  5. Provide the current valid password for the account and confirm.

Once the task completes successfully, SDDC Manager will fetch the newly supported expiration date, the database will update successfully, and the account status will reflect as Connected.

Additional Information

Powered by Wolken Software