Unable to start the Inventory Services
search cancel

Unable to start the Inventory Services

book

Article ID: 301191

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:
  • Unable to start the Inventory Services.
     
  • You see the below errors:

2017-03-30T20:52:20.579Z State for service invsvc: STOPPED
2017-03-30T20:53:00.620Z Error waiting for service invsvc start. Exit code: 1066 Current State: 1ERROR:root:Unable to start service invsvc, Exception: {"resolution": null,"detail": [{"args": ["invsvc"],"id": "install.ciscommon.service.failstart","localized": "An error occurred while starting service 'invsvc'","translatable": "An error occurred while starting service '%(0)s'"}],"componentKey":null,"problemId": null}Unable to start service invsvc, Exception: {"resolution": null,"detail": [{"args": ["invsvc"],"id": "install.ciscommon.service.failstart","localized": "An erroroccurred while starting service 'invsvc'",
"translatable": "An error occurred while starting service '%(0)s'"}],"componentKey": null,"problemId": null}Invsvc.log (Scenario 1)Caused by: m.vmware.vim.query.server.kvlocal.impl.exception.KVInternalException: Unable to populate existing providers:at m.vmware.vim.query.server.kvlocal.impl.VCDBKVManagerImpl.init(VCDBKVManagerImpl.java:76)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)... 84 moreCaused by: com.vmware.vim.query.server.kvlocal.impl.exception.KVExecutionException: SQL Error:at com.vmware.vim.query.server.kvlocal.impl.KVUtil.createAndLogExecutionException(KVUtil.java:153)atcom.vmware.vim.query.server.kvlocal.impl.provider.VCDBProviderFactory.getConnection(VCDBProviderFactory.java:116)... 91 moreCaused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Login failed for user 'CORPORATE\PDC-VCENTER01$'. ClientConnectionId:17edb35c-d8ab-4950-8846-284beb2afb50)at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)... 94 moreCaused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'CORPORATE\PDC-VCENTER01$'. ClientConnectionId:17edb35c-d8ab-4950-8846-284beb2afb50invsvc.log [Scenario 2]Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'localKvManager' defined in class path resource [server/config/query-server-config.xml]: Invocation of init method failed; nested exception is com.vmware.vim.query.server.kvlocal.impl.exception.KVInternalException: Unable to populate existing providers:at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)... 75 moreCaused by: com.vmware.vim.query.server.kvlocal.impl.exception.KVInternalException: Unable to populate existing providers:at com.vmware.vim.query.server.kvlocal.impl.VCDBKVManagerImpl.init(VCDBKVManagerImpl.java:76)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)... 84 moreCaused by: com.vmware.vim.query.server.kvlocal.impl.exception.KVExecutionException: SQL Error:at com.vmware.vim.query.server.kvlocal.impl.KVUtil.createAndLogExecutionException(KVUtil.java:153)at com.vmware.vim.query.server.kvlocal.impl.VCDBKVManagerImpl.init(VCDBKVManagerImpl.java:72)... 91 moreCaused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Cannot open database "vCenter" requested by the login. The login failed. ClientConnectionId:3eba6484-4358-4e46-b07d-55bb87337021)at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)at com.vmware.vim.query.server.kvlocal.impl.provider.VCDBProviderFactory.getConnection(VCDBProviderFactory.java:94)... 94 moreCaused by: com.microsoft.sqlserver.jdbc.SQLServerException: Cannot open database "vCenter" requested by the login. The login failed. ClientConnectionId:3eba6484-4358-4e46-b07d-55bb87337021at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:254)

  • In invsvc.log, you see the below (Log Snippets in vCenter Appliance):

Caused by: com.vmware.vim.query.server.kvlocal.impl.exception.KVInternalException: Unable to populate existing providers:
at com.vmware.vim.query.server.kvlocal.impl.VCDBKVManagerImpl.init(VCDBKVManagerImpl.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
... 84 more
Caused by: com.vmware.vim.query.server.kvlocal.impl.exception.KVExecutionException: SQL Error:
at com.vmware.vim.query.server.kvlocal.impl.KVUtil.createAndLogExecutionException(KVUtil.java:153)
at com.vmware.vim.query.server.kvlocal.impl.provider.VCDBProviderFactory.getConnection(VCDBProviderFactory.java:116)
... 91 more
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (FATAL: the database system is starting up)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
... 94 more
Caused by: org.postgresql.util.PSQLException: FATAL: the database system is starting up
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:276)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)

  • In vpostgres.log, you see the below logs:


2017-03-30 13:17:26.909 PDT 58caa140.b60 0 VCDB vc FATAL: terminating connection due to administrator command
2017-03-30 13:17:26.909 PDT 58caa013.149c 0 LOG: autovacuum launcher shutting down
2017-03-30 13:17:26.937 PDT 58caa013.149c 0 ERROR: canceling statement due to user request
2017-03-30 13:17:26.937 PDT 58caa013.149c 0 LOG: autovacuum launcher shutting down
2017-03-30 13:17:26.944 PDT 58caa013.14ac 0 ERROR: canceling statement due to user request
2017-03-30 13:17:26.944 PDT 58caa013.14ac 0 CONTEXT: SQL statement "SELECT 1;"
2017-03-30 13:17:30.855 PDT 58dd67da.f760 0 VCDB vc FATAL: the database system is shutting down
2017-03-30 13:17:31.074 PDT 58dd67db.eda4 0 VCDB vc FATAL: the database system is shutting down

  • In Invsvc.log, you see the below logs (Log Snippets for Windows VC)

Caused by: java.io.IOException: SQL Server did not return a response. The connection has been closed. ClientConnectionId:e2359597-0751-4d20-9bf5-e633da73f8ba
at com.microsoft.sqlserver.jdbc.TDSChannel$SSLHandshakeInputStream.ensureSSLPayload(IOBuffer.java:651)
at com.microsoft.sqlserver.jdbc.TDSChannel$SSLHandshakeInputStream.readInternal(IOBuffer.java:708)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory
(The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer
(SSL) encryption. Error: "SQL Server did not return a response. The connection has been closed. ClientConnectionId:e2359597-0751-4d20-9bf5-e633da73

 

Note: This log excerpt is an example. Date, time, and environmental variables may vary depending on your environment.


Environment

VMware vSphere ESXi 6.0

Resolution

Before proceeding with troubleshooting, ensure to validate the following:

  1. Check the ODBC connection.
  2. Check the DB access for the account where the vCenter Services is started.
  3. Ensure if the vCenter service and Inventory service is started with same logon account.
  4. Reset the database password using vpxd -p if the ODBC connection is changed with different DB account.
  5. Increase the heap size for the Inventory Services.
  6. Check the c:\programdata\vmware\vCenterServer\cfg\vmware-vpx\vpxd.cfg file for the key value to make sure if the SQL account is correct.
  7. Check the c:\programdata\vmware\vCenterServer\cfg\vmware-vpx\vcdb.properties and update the correct user name and password used in the ODBC connection.

Scenario 1:

If you see 'SQL Error' followed by 'Login failed for user' in the logs:

- Check if ODBC connection is working.

- Ensure you have the vCenter Server and the Inventory services starting with the same account which has access to vCenter database.

Scenario 2:

If you see 'SQL Error' followed by 'Cannot open database <DSN Name> requested by the login. The login failed' in the logs:

- Ensure you have the DB access for the account which vCenter Service is started. If you change the logon account for the vCenter Services, change the logon account for the Inventory Services with the same account.

Scenario 3:

If you see 'SQL Error' followed by 'PSQLException: FATAL: the database system is starting up' in the logs:

  • ODBC connection fails during Test connection.

    >Postgres Database: Ensure you have a working ODBC connection. ODBC driver 'org.postgresql.Driver' might be corrupted, hence restart the postgres services and verify the connection.

Scenario 4:

If you see 'SQL Error' followed by 'SQL Server did not return a response. The connection has been closed' or ' org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption. Error: "SQL Server did not return a response. The connection has been closed' in the logs:

  • ODBC connection fails.

>SQL Database: SQL Server Native client driver is having any issue's, might need to reinstall the SQL Server Native Client. The issue is with the jdbc driver version 1.7.
 

To workaround this issue, re-enable TLSv1.0 on the SQL server and reset the database password using vpxd -p and check the ODBC connection.


Additional Information

Inventory Service を開始できない
简体中文：无法启动 Inventory Service

Powered by Wolken Software