Initial configuration fails when configuring Replicator Service instances in Cloud Director Availability

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• Configuring Replicator Service instances during the initial configuration of the Cloud Service fails with a connection refused error over port 8043 to the Replicator Service instances.
• In the /opt/vmware/h4/manager/log/manager.log file on the Cloud Replication Management Appliance, you see entries similar to:

2021-01-20 08:29:53.229 ERROR - [UI-########-####-####-####-########84da-Qe] [https-jsse-nio-8443-exec-6] c.v.h.c.c.error.ExceptionAdvisorBase     : A POST request from root[10.X.X.X] to /replicators failed.

com.vmware.h4.api.error.exceptions.client.ConnectionRefusedException: Connection refused.
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at com.vmware.h4.api.error.GenericServerExceptionProvider.get(GenericServerExceptionProvider.java:118)
        ...

• In the /opt/vmware/h4/replicator/log/replicator.log on the Cloud Replicator Appliance, you see entries similar to:

2021-01-20 09:33:49.037 ERROR - [UI-########-####-####-####-########6d36-b6-1J-Mr] [https-jsse-nio-8043-exec-7] c.v.h.c.c.error.ExceptionAdvisorBase     : A POST request from root[cloud_VM_ip] to /managers failed.

com.vmware.exception.ConnectionRefusedException: Connect to 127.0.0.2:8044 [/127.0.0.2] failed: Connection refused (Connection refused)
        at com.vmware.exception.converter.ClientExceptionConverter.convertException(ClientExceptionConverter.java:45)
        at com.vmware.rest.client.AbstractRestClient.genericExchange(AbstractRestClient.java:158)
        at com.vmware.rest.client.json.RestClient.exchange(RestClient.java:97)
        ...

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware Cloud Director Availability 4.x

Cause

This issue can occur if the Cloud Replication Management Appliance was booted before it was connected to a network or failed to have its IP set by dhclient on startup.

This can cause the Manager Service endpoint being set to a loopback address so, when the Manager Service endpoint is sent to the Replicator Service instance, the Replicator Service is unable to connect back to the Manager Service as it is attempting to connect via the loopback address.

Resolution

To verify you are experiencing this issue, perform the following steps:

1. SSH to the Cloud Replication Management Appliance and log in as root.
2. Authenticate against the Manager Service with the following command:

h4 loginroot 'password'

3. Verify the configured Manager Service endpoint is set to a loopback address:

h4 endpoints

4. Verify the OS of the Cloud Replication Management Appliance has the correct IP set:

ifconfig

To resolve this issue, reboot the Cloud Replication Management Appliance for the active IP address to be applied to the Manager Service endpoint.

Additional Information

If you need to change the IP address of a Cloud Director Availability appliance, see the Configure a Network Adapter section of the Cloud Director Availability documentation.