VM restoration failure: Port 111 conflict with third-party software
search cancel

VM restoration failure: Port 111 conflict with third-party software

book

Article ID: 437938

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Note: While this article specifically addresses a conflict between a third-party backup solution and other RPC-based services, the diagnostic steps and logic apply to any scenario where a third-party application or service occupies TCP Port 111 on a Windows-based storage proxy.

When attempting to restore a Virtual Machine using orchestration or backup tools, the task fails because the NFS backup service cannot establish a connection. This typically occurs on Windows-based storage proxies where TCP Port 111 (RPC Portmapper) is already occupied by another service. Following errors are observed: 

  • Failed to connect to NFS service on host <Host FQDN>, port '6161'
  • Unable to check if NFS is mounted on the host
  • No connection could be made because the target machine actively refused it
  • Unable to establish connection: timeout reached

The ESXi logs will record repeated connectivity failures.

/var/log/vmkernel.log: vmkernel logs record multiple WARNING: NFS: 2834: Failed to get attributes (No connection) and NFS: 2937 entries, indicating an inability to establish an RPC session.

YYYY-MM-DDThh:mm Wa(180) vmkwarning: cpu26:#####)WARNING: NFS: 2834: Failed to get attributes (No connection)
YYYY-MM-DDThh:mm In(182) vmkernel: cpu26:#####)NFS: 2937: [Repeated 3 times] Failed to get object (0x4539##a9aa16) ## ##### ##### #### 0 0 0 0 0 0 0 0 :No connection
YYYY-MM-DDThh:mm In(182) vmkernel: cpu26:#####)NFS: 2944: Failed to get object (0x45392df1aa16) ## ##### ##### #### 0 0 0 0 0 0 0 0 :No connection
YYYY-MM-DDThh:mm Wa(180) vmkwarning: cpu10:55545544)WARNING: NFS: 2834: Failed to get attributes (No connection)
YYYY-MM-DDThh:mm Wa(180) vmkwarning: cpu10:55545544)WARNING: NFS: 2834: Failed to get attributes (No connection)
YYYY-MM-DDThh:mm Wa(180) vmkwarning: cpu10:55545544)WARNING: NFS: 2834: Failed to get attributes (No connection)
YYYY-MM-DDThh:mm Wa(180) vmkwarning: cpu10:55545544)WARNING: NFS: 2834: Failed to get attributes (No connection)
 
/var/log/sdrsInjector logs indicate the Storage DRS poll the volume state, indicating the datastore is inaccessible to management services
2026-04-15T14:49:07.771Z Er(11) sdrsInjector[2098707]: Could not get state for NFS volume #####
2026-04-15T14:49:39.775Z Er(11) sdrsInjector[2098707]: Could not get state for NFS volume #####
2026-04-15T14:50:11.780Z Er(11) sdrsInjector[2098707]: Could not get state for NFS volume #####
2026-04-15T14:50:43.784Z Er(11) sdrsInjector[2098707]: Could not get state for NFS volume #####
 
/var/log/hostd.log: The Operation not supported (11) error indicates that there was a failed attempt to access the .vmx file on the backup datastore.
YYYY-MM-DDThh:mm Wa(164) Hostd[2098861]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore/Restored VM Name/Restored VM Name.vmx opID=WorkQueue-####-d1-#### sid=#### user=vpxuser:<no user>] File - failed to get objectId,
 '/vmfs/volumes/datastore/Restored VM Name/Restored VM Name.vmx': Operation not supported (11)
YYYY-MM-DDThh:mm Wa(164) Hostd[2098894]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/datastore/Restored VM Name/Restored VM Name.vmx opID=WorkQueue-####-d1-#### sid=#### user=vpxuser:<no user>] File - failed to get objectId, '/vmfs/volumes/datastore/Restored VM Name/Restored VM Name.vmx': Operation not supported (11)

Environment

VMware vCenter Server

Cause

The backup service requires TCP Port 111 to negotiate connections with ESXi hosts. This issue occurs when another third-party software agent binds to Port 111 first. Because the port is already in use, the RPC-based backup service cannot initialize its listener. When the ESXi host attempts to connect, the connection is actively refused by the third-party service, preventing the datastore from mounting or the restored VM from being accessed.

Resolution

  1. Identify the Process Holding Port 111: Open PowerShell as an Administrator on the Backup Proxy/Repository and run the following command to find the OwningProcess ID:

    PS <drive>:\Program Files (x86)\...> Get-NetTCPConnection -LocalPort 111 | Select-Object OwningProcess

    Example Output showing PID ####:

    OwningProcess-------------         ####

  2. Identify the Conflicting Application: Use the PID identified in Step 1 to reveal the process name:

    Get-Process -Id [Enter_PID_Here]

    Example output confirming the conflict with a third-party process:

    Handles  NPM(K)  PM(K)  WS(K)  CPU(s) Id    SI  ProcessName
    -------  ------  -----  -----  ------ --    --  -----------
        14#   1#     104##  9##8   #.02   ##64  0   [Third-party Process]

    Note: In this example, the conflict is observed with a third-party backup or monitoring service. Port 111 may be utilized by various external utilities. Always verify the service identity before disabling or modifying its configuration.

  3. Remediate the Conflict: If the Process is identified as a non-essential service for current operation, stop and disable that service.

  4. Restart Services: Restart the NFS backup Service to allow it to bind to Port 111

  5. Verification: Run Get-NetTCPConnection -LocalPort 111 again to ensure the port is now owned by the correct process. On the ESXi host, run esxcfg-nas -r and retry the VM restoration.

Powered by Wolken Software