The vCenter Content Library is failing to synchronize or upload files because a network or service-level connection is being explicitly rejected (Connection Refused) when attempting to write data to the target datastore.

Products

• VMware vCenter Server

Issue/Introduction

Content Library fails to sync with connection refused error.

cls.log:

YYYY-MM-DDT05:12:48.908Z | WARN     | mhw1bi8c-1103298-auto-nnb7-h5:70208104-c6-fb | cls-simple-activity-7     | ApiMethodSkeleton              | Implementation method reported unexpected exception: com.vmware.vapi.std.errors.InternalServerErrorcom.vmware.vapi.std.errors.InternalServerError: InternalServerError (com.vmware.vapi.std.errors.internal_server_error) => {
  messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {

    id = com.vmware.vdcs.cls-main.error_while_transferring_to,    defaultMessage = Error transferring file VMware-XXX.iso to ds:///vmfs/volumes/XXX//contentlib-XXX/XXX/VMware-VLR-XXX.iso?serverId=19d9325e-ad1e-43f0-bade-06d5040fc2c2. Reason: Error while uploading to datastore (ds:///vmfs/volumes/XXX//contentlib-XXX/VMware-VLR-XXX.iso?serverId=XXX): Connection refused (Connection refused),  args = [VMware-VLR-9.0.5.0.25103105.iso, ds:///vmfs/volumes/XXX//contentlib-XXX/VMware-VLR-XXX.iso?serverId=XXX, Error while uploading to datastore (ds:///vmfs/volumes/XXX//contentlib-XXX/XXX/VMware-VLR-XXX.iso?serverId=XXX

c2): Connection refused (Connection refused)],

    params = <null>,

    localized = <null>

}],

    data = <null>,

    errorType = INTERNAL_SERVER_ERROR

}

.

.

Error while uploading to datastore (ds:///vmfs/volumes/XXX//contentlib-XXX/XXX/VMware-VLR-XXX.iso? serverId=XXX): Connection refused (Connection refused),
Error while uploading to datastore  Connection refused (Connection refused)

 

 

Environment

vCenter Server 8.x

Cause

The Content Library Service (CLS) encountered an InternalServerError during a file transfer activity. Specifically, the API method skeleton reported a com.vmware.vapi.std.errors.internal_server_error. The root symptom is a TCP Connection Refused error occurring at the transport layer between the vCenter Server (which manages the Content Library orchestration) and the target ESXi host or the storage provider service responsible for datastore I/O.

 

As the message in logs states Connection refused, we need to check the Ports and connectivity between Publisher and Subscriber vCenter server.

[root@esx01:~] nc -zuv X.X.X.X 902
Connection to X.X.X.X 902 port [udp/*] succeeded!

[root@esx01:~] nc -zv X.X.X.X 902
No output as Port TCP 902 is disabled.

[root@esx01:~] nc -zv X.X.X.X 80
Connection to X.X.X.X port [tcp/http] succeeded!

[root@esx01:~] nc -zv X.X.X.X 443
Connection to X.X.X.X 443 port [tcp/https] succeeded!

Note: For required ports enablement refer KB https://knowledge.broadcom.com/external/article/318895/port-requirements-for-vmware-vsphere-esx.html  

Resolution

Enable required ports and Sync the Content Library.