Fleet Management fails to load SDDC Inventory information of Passwords and Certificates
search cancel

Fleet Management fails to load SDDC Inventory information of Passwords and Certificates

book

Article ID: 419281

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

  • VMware Cloud Foundation Operations UI fails to load SDDC information of Certificates, Passwords and Tasks. The UI keeps loading with no error or failure message as shown below

 

  • The UI also shows warning banner about certificates as "Your certificate is expired. Please check the expiry date to take action"
  • SDDC UI load inventory information successfully. No references to certificate expiry are shown.
  • Log snippets provide key information about error/exceptions drawn:

/var/log/vmware/vcf/domainmanager/domainmanager.log

YYYY-MM-DDTHH:MIN:SEC INFO  [vcf_dm,691212284f45138d0c9fa9a4f989b482,c012] [c.v.v.q.impl.util.WorkerThreadPool,vac-scheduler-1] Thread pool metrics: [pool size = 0, largest pool size = 8, active threads = 0, queued tasks = 0, total tasks = 25676, completed tasks = 25676]
YYYY-MM-DDTHH:MIN:SEC INFO  [vcf_dm,691212284f45138d0c9fa9a4f989b482,c012] [c.v.v.q.i.util.QueryServiceMetrics,vac-scheduler-1] QueryService metrics: [total queries = 0, with 50+ / 100+ / 200+ ids = 0 / 0 / 0, for 10+ / 20+ / 40+ data sources = 0 / 0 / 0, above 1+ / 2+ / 4+ secs = 0 / 0 / 0, with errors = 0]
YYYY-MM-DDTHH:MIN:SEC INFO  [vcf_dm,691212284f45138d0c9fa9a4f989b482,c012] [c.v.v.query.impl.session.SessionPool,vac-scheduler-1] Session pool metrics: [pool size = 1, evictions = 0, hit rate = 1.0, load failure rate = 0.0]
YYYY-MM-DDTHH:MIN:SEC DEBUG [vcf_dm,69121237b2e0afb49f159c7bcfb38b51,d1ec] [c.v.v.d.s.i.DomainCapabilitiesServiceImpl,vac-scheduler-1]  Get domain Capabilities domainIds null, capabilities null, fetchCapabilitiesInParallel false
YYYY-MM-DDTHH:MIN:SEC ERROR [vcf_dm,0000000000000000,0000] [o.s.s.s.TaskUtils$LoggingErrorHandler,vac-scheduler-1]  Unexpected error occurred in scheduled task
com.vmware.evo.sddc.common.services.error.SddcManagerServicesIsException: Workload Domain Details Not Found.
        at com.vmware.evo.sddc.common.services.adapters.inventoryservice.InventoryServiceAdapterImpl.getDomainsData(InventoryServiceAdapterImpl.java:202)
        at com.vmware.vcf.domaincapabilities.service.impl.DomainCapabilitiesServiceImpl.getDomainCapabilities(DomainCapabilitiesServiceImpl.java:57)
Caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException: java.net.SocketTimeoutException: Connect timed out
        at com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiClient.execute(ApiClient.java:711)


/var/log/vmware/vcf/operationsmanager/operationsmanager.log

YYYY-MM-DDTHH:MIN:SEC INFO  [vcf_om,6912122ee27f92baa22c66aa9c05d3a3,7570] [c.v.v.query.impl.session.SessionPool,pool-8-thread-1] Session pool metrics: [pool size = 1, evictions = 0, hit rate = 1.0, load failure rate = 0.0]
YYYY-MM-DDTHH:MIN:SEC INFO  [vcf_om,0000000000000000,0000] [c.v.v.t.services.TaskPublisher,om-scheduler-3] Querying completed tasks after YYYY-MM-DDTHH:MIN:SEC
YYYY-MM-DDTHH:MIN:SEC WARN  [vcf_om,0000000000000000,0000] [c.v.v.t.services.TaskPublisher,om-scheduler-3] Failed to update vCenter tasks com.vmware.vcf.rest.api.runtime.ApiException: java.net.SocketTimeoutException: Connect timed out
        at com.vmware.vcf.rest.api.runtime.ApiClient.execute(ApiClient.java:711)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.net.SocketTimeoutException: Connect timed out
        at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:551)
        at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:602)
        ... 11 common frames omitted

/var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log

YYYY-MM-DDTHH:MIN:SEC ERROR [common,6912121c2444d3bf0991fa1f1c5fe42e,0bff] [c.v.v.i.sync.utils.VcSyncManagerUtil,cs-exec-8] Error connecting to vCenter <vCenter FQDN>, with exception {} 
com.vmware.vcf.secure.config.SecurityConfigException: Fail to retrieve security configuration.
        at com.vmware.vcf.secure.config.sddc.SecurityConfigurationServiceImpl.isFipsModeEnabled(SecurityConfigurationServiceImpl.java:61)
        at com.vmware.vcf.secure.config.sddc.SecurityConfigurationServiceImpl.getCipherSuites(SecurityConfigurationServiceImpl.java:67)
Caused by: com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiException: java.net.SocketTimeoutException: Connect timed out
        at com.vmware.cloud.foundation.rest.commonsvcs.runtime.ApiClient.execute(ApiClient.java:711)
        ... 12 common frames omitted
Caused by: java.net.SocketTimeoutException: Connect timed out
        at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:551)
        ... 15 common frames omitted

  • On VCF Operations, error occurred on VCF Adapter with warning "An error occurred while collecting data. Check the logs for additional"

Environment

VMware Cloud Foundation 9.0

Resolution

To address the issue, please restart SDDC manager services by following below mentioned steps. The order of service restart needs to be followed to fix the issue.

  • Login as 'VCF' user to SDDC manager SSH session
  • Switch to root user using 'su' command
  • Restart services in order:
    • systemctl restart postgres
    • systemctl restart commonsvcs
    • systemctl restart domainmanager
    • systemctl restart operationsmanager
  • VCF Operations should now be able to load SDDC inventory information
Powered by Wolken Software