Imaging a node using VIA fails with an error "[ERROR] Failed to obtain from PT Agent :Failed to get disk information via PT Agent Service"

Products

VMware Cloud Foundation

Issue/Introduction

Symptoms:
Note: This issue is applicable only for VMware Cloud Foundation 2.3.x deployments on Dell/EMC VxRack SDDC or VxRail.

VIA fails to image the node.
You see errors similar to the following in the /var/log/evorack-imaging/via.log file on the VIA VM :

2018-07-26 04:51:43,909 [44 ESXI_SERVER 192.168.100.59] DEBUG: com.vmware.via.util.SshUtil - End of execution of command [/opt/vxrail/bin/dell_disk_info.sh -u root -p <PASSWORD>]

2018-07-26 04:51:43,910 [44 ESXI_SERVER 192.168.100.59] ERROR: com.vmware.via.esxi.intraimaging.tasks.ESXiStorageConfigurationTask - Command '/opt/vxrail/bin/dell_disk_info.sh -u root -p <PASSWORD>' failed with result

'VXRACK_DISKGROUP_MAPPING_STATUS :: SCRIPT EXECUTION SUCCEEDED :: CODE#0 :: pre-chekcing was successful!

Prepare to update JSON file .capacityflash.json

[WARNING] Disks are not in eligible status found: t10.ATA_____SATADOM2DML_3SE__________________________TW00T4M4482937AP1034

2018-07-26 04:51:43,910 [44 ESXI_SERVER 192.168.100.59] ERROR: com.vmware.via.esxi.intraimaging.tasks.ESXiStorageConfigurationTask - Task Storage Configuration failed.

2018-07-26 04:51:43,910 [44 ESXI_SERVER 192.168.100.59] ERROR: com.vmware.via.esxi.intraimaging.tasks.ESXiStorageConfigurationTask - Unable to execute command /opt/vxrail/bin/dell_disk_info.sh -u root -p <PASSWORD>

2018-07-26 04:51:43,916 [44 ESXI_SERVER 192.168.100.59] ERROR: com.vmware.via.imaging.SetupActivity - Failed task Storage Configuration for device type=ESXI_SERVER ip=192.168.100.59

2018-07-26 04:51:58,593 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.service.impl.ImagingMonitoringServiceImpl - In last 60 seconds, not found imaging in-progress for any device. Imaging will timeout in 840 seconds

2018-07-26 04:51:58,593 [pool-11-thread-1] INFO : com.vmware.vrack.via.service.impl.ImagingMonitoringServiceImpl - Imaging time out occured for run id 1. Hence stopping imaging..

2018-07-26 04:51:58,594 [pool-11-thread-1] INFO : com.vmware.vrack.via.imaging.service.impl.ImagingServiceImpl - Stopping imaging for image id 5b59451ea4eae20a34a2f2b4 and runId 1.

2018-07-26 04:51:58,594 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.imaging.service.impl.ImagingServiceImpl - Imaging activity StartImagingActivity is already completed.

2018-07-26 04:51:58,594 [pool-11-thread-1] INFO : com.vmware.vrack.via.microservice.manager.ImagingServiceManager - Sending stop imaging request to all imaging services for RunId 1 .

2018-07-26 04:51:58,613 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.microservice.manager.ImagingServiceManager - Successfully stopped imaging service http://192.168.100.2:8081 for runId 1

2018-07-26 04:51:58,623 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.microservice.manager.ImagingServiceManager - Successfully stopped imaging service http://192.168.100.2:8082 for runId 1

2018-07-26 04:51:58,624 [pool-11-thread-1] INFO : com.vmware.vrack.via.dhcpd.DHCPServer$DHCPServerListenerTask - Stopping DHCP server on port 67

2018-07-26 04:51:58,624 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.dhcpd.DHCPServer - DHCP service is stopped

2018-07-26 04:51:58,624 [pool-11-thread-1] DEBUG: com.vmware.vrack.via.imaging.service.impl.ImagingServiceImpl - Correcting device activities status..

2018-07-26 04:51:58,624 [SimpleAsyncTaskExecutor-1] INFO : com.vmware.vrack.via.dhcpd.DHCPServer$DHCPServerListenerTask - Exiting from DHCP server listening mode.

2018-07-26 04:51:58,624 [SimpleAsyncTaskExecutor-1] INFO : com.vmware.vrack.via.dhcpd.DHCPServer$DHCPServerListenerTask - DHCP Server is stopped

2018-07-26 04:51:58,627 [pool-11-thread-1] WARN : com.vmware.vrack.via.imaging.service.impl.ImagingServiceImpl - Correcting imaging status from IMAGING_PHASE_RUNNING to IMAGING_PHASE_STOPPED for Run id: 1

2018-07-26 04:51:58,629 [pool-11-thread-1] INFO : com.vmware.vrack.via.service.impl.ImagingMonitoringServiceImpl - Imaging time out service completed for run id 1.

2018-07-26 04:51:58,599 [http-nio-8081-exec-2] DEBUG: com.vmware.via.controller.CommonImagingController - Stop Imaging!, id = 5b59451ea4eae20a34a2f2b4

2018-07-26 04:51:58,607 [http-nio-8081-exec-2] INFO : com.vmware.via.controller.CommonImagingController - Stopping Imaging for run ID 1

2018-07-26 04:51:58,618 [http-nio-8082-exec-9] DEBUG: com.vmware.via.controller.CommonImagingController - Stop Imaging!, id = 5b59451ea4eae20a34a2f2b4

2018-07-26 04:51:58,620 [http-nio-8082-exec-9] INFO : com.vmware.via.controller.CommonImagingController - Stopping Imaging for run ID 1

[ERROR] Failed to obtain from PT Agent :Failed to get disk information via PT Agent Service. Host [fe80::266e:96ff:fe97:9b90%vmk0], Port 8086

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware Cloud Foundation 2.3.x

Resolution

This is a known issue affecting VxRack SDDC or VxRail with VMware Cloud Foundation 2.3.x. Currently there is no resolution.

Workaround:
Follow the workaround below to resume imaging of the node:

Restart the iDRAC on the node whose imaging was failed. Refer to server documentation for detailed instructions.
Login to the host through the web-based host client at https://<host IP>. Login as root with the password EvoSddc!2016.
Select Manage and then click on the Services tab. Right-click the ptagent service and select Restart.
Make sure that both vxrail-platform-service & vxrail-platform-service-util services are running on the node.
In the VIA UI, click the Resume button if imaging does not automatically resume.