Error "com.vmware.vcloud.common.persist.exception.CellNotActiveException: Cell is not active" while accessing the Provider and Tenant Portals after a failover from the primary cell to a standby cell

Products

• VMware Cloud Director

Issue/Introduction

• After a failover from the primary cell to a standby cell, the Provider and Tenant Portals become inaccessible and throws an error: HTTP ERROR 404 JSP file [/error.jsp] not found

HTTP ERROR 404 JSP file [/error.jsp] not found

URI: /provider/

STATUS: 404

MESSAGE: JSP file [/error.jsp] not found

SERVLET: jsp

CAUSED BY: com.vmware.vcloud.common.persist.exception.CellNotActiveException: Cell is not active

Caused by:

com.vmware.vcloud.common.persist.exception.CellNotActiveException: Cell is not active

at com.vmware.vcloud.common.persist.impl.CellLivenessStatusServiceImpl.validateIsActive(CellLivenessStatusServiceImpl.java:81)

at com.vmware.vcloud.common.persist.QueryInterceptor.validateIsActive(QueryInterceptor.java:118)

at com.vmware.vcloud.common.persist.QueryInterceptor.onPrepareStatement(QueryInterceptor.java:170)

• The embedded database cluster health shows as Healthy following the switchover.
• FIPS mode is set to disabled on all the cells.
• Following the failover, the /opt/vmware/vcloud-director/logs/cell-management-tool.log shows the following entries:

YYYY-MM-DD 07:23:00,669 | ERROR    | main                      | PGjdbcHostnameVerifier         | Server name validation failed: certificate for host <eth1 ip_address of the cell> dNSName entries subjectAltName, but none of them match. Assuming server name validation failed |
YYYY-MM-DD 07:23:00,673 | ERROR    | main                      | ReconfigureDatabaseCommand     | Invalid database configuration: The hostname <eth1 ip_address of the cell> could not be verified by hostnameverifier PgjdbcHostnameVerifier. |
org.postgresql.util.PSQLException: The hostname <eth1 ip_address of the cell> could not be verified by hostnameverifier PgjdbcHostnameVerifier.
        at org.postgresql.ssl.MakeSSL.verifyPeerName(MakeSSL.java:93)
        at org.postgresql.ssl.MakeSSL.convert(MakeSSL.java:62)
  ...

YYYY-MM-DD 07:23:00,674 | ERROR    | main                      | ReconfigureDatabaseCommand     | Invalid database configuration provided. Aborting configuration of local cell. |
YYYY-MM-DD 07:23:00,674 | DEBUG    | main                      | ReconfigureDatabaseCommand     | Starting reconfigure of remote cells |
YYYY-MM-DD 07:23:00,894 | DEBUG    | main                      | SslContextImpl                 | SecurityContextOrgIdAccessor not yet set - will provide a new/uncached SSLContext |
YYYY-MM-DD 07:23:00,904 | ERROR    | main                      | PGjdbcHostnameVerifier         | Server name validation failed: certificate for host <eth1 ip_address of the cell> dNSName entries subjectAltName, but none of them match. Assuming server name validation failed |
YYYY-MM-DD 07:23:00,906 | ERROR    | main                      | JDBCExceptionReporter          | The hostname <eth1 ip_address of the cell> could not be verified by hostnameverifier PgjdbcHostnameVerifier. |
YYYY-MM-DD 07:23:00,906 | ERROR    | main                      | CellManagementTool             | Error executing command: org.hibernate.exception.JDBCConnectionException: Cannot open connection |
java.lang.RuntimeException: org.hibernate.exception.JDBCConnectionException: Cannot open connection
        at com.vmware.vcloud.common.install.cmt.ReconfigureDatabaseCommand.doCommand(ReconfigureDatabaseCommand.java:224)
Caused by: org.hibernate.exception.JDBCConnectionException: Cannot open connection
        at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:97)

Environment

VMware Cloud Director 10.6.x

Cause

This issue occurs when the PostgreSQL certificate Subject Alternative Name (SAN) does not include the eth1 ip address of the cell.

Resolution

Appliances which run with the default self-signed certificates for PostgreSQL are not affected.

To resolve the issue, generate new certificate for postgres with all the below entries in SAN field and replace the certificate on all the VMware Cloud Director Cells: Replace or Renew the VMware Cloud Director Appliance Management Certificates

• Cell FQDN
• Cell short name
• Eth0 IP Address
• Eth1 IP Address