Polling of ESXi host from Dell PowerPath Management Appliance returns "PowerPath Management Appliance is unable to communicate with the host"

Products

• VMware vCenter Server

Issue/Introduction

• Attempting to poll ESXi host from Dell PowerPath Management Appliance (PPMA) fails with "PowerPath Management Appliance is unable to communicate with the host"

• /var/run/log/syslog.log (on ESXi host)

YYYY-MM-DDTHH:MM:SS Wa(84) sfcb-CIMXML-Processor[PID]: pam_unix(sfcb:auth): check pass; user unknown
YYYY-MM-DDTHH:MM:SS Er(27) sfcb-CIMXML-Processor[PID]: [module:pam_lsass]pam_do_authenticate: error [login:<session ID>][error code:2]
YYYY-MM-DDTHH:MM:SS Er(27) sfcb-CIMXML-Processor[PID]: [module:pam_lsass]pam_sm_authenticate: failed [error code:2]
YYYY-MM-DDTHH:MM:SS Wa(84) sfcb-CIMXML-Processor[PID]: pam_unix(sfcb:auth): check pass; user unknown
YYYY-MM-DDTHH:MM:SS Er(27) sfcb-CIMXML-Processor[PID]: [module:pam_lsass]pam_do_authenticate: error [login:<session ID>][error code:2]
YYYY-MM-DDTHH:MM:SS Er(27) sfcb-CIMXML-Processor[PID]: [module:pam_lsass]pam_sm_authenticate: failed [error code:2]
YYYY-MM-DDTHH:MM:SS Er(27) sfcb-ProviderManager[PID]: lookupProviderList: lookupProviders(ns=powerpath/cimv2,class=Powerpath_RegisteredProfile,type=0x1) returned rc = 3

Cause

This issue stems from a discrepancy between the PPMA authentication token and the corresponding ESXi user session ticket

In order to validate the vSphere functionality, execute the below steps

Option 1: Use the <login user ID> reported from the host log file (syslog.log)

• Log in to VCSA via ssh
• Execute the below command to validate the authentication

curl -vk 'https://<ESXi host IP/FQDN>:5989' --request POST --basic --user <session ID>

Note: The value for the <session ID> is captured under syslog.log on ESXi host.

Sample: 

YYYY-MM-DDTHH:MM:SS Er(27) sfcb-CIMXML-Processor[PID]: [module:pam_lsass]pam_do_authenticate: error [login:<session ID>][error code:2]

Option 2: Generate the ticket using VCSA MOB and validate the functionality

• Log on to vCenter Server Appliance(VCSA) Managed Object Browser (MOB) using SSO Administrator credentials
• Click on content

  • Under type ManagedObjectReference:Folder click group-d1 (Datacenters)
  • Under type ManagedObjectReference:ManagedEntity[] select datacenter-<ID> (datacenter name)
  • Under hostFolder click group-h# (host)
  • Under ManagedObjectReference:ManagedEntity[] click domain-c# (<Cluster Name>)
  • Under ManagedObjectReference:HostSystem[] click on the impacted host
  • Under Methods click on AcquireCimServicesTicket
    
    
    • Click Invoke Method
    • copy the sessionId value
      
      
• Log in to VCSA via ssh
• Execute the below command to validate the authentication

curl -vk 'https://<ESXi host IP/FQDN>:5989' --request POST --basic --user <session ID>

Note: The Session ID is valid for 120 seconds post the invoke 

Resolution

Proceed to engage Dell support to identify the underlying cause of the mismatch as this has been reported to be resolved with newer version of PPMA.