Deploying VCH to cluster nested inside folders results in the Portlayer failing with message FATAL configure_port_layer ERROR: Failure finding host (): no default host found"

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
While deploying VCH using the < vic-machine-OS create > utility on a nested cluster inside folders results in the Port layer fails with error as below or similar:

< FATAL configure_port_layer ERROR: Failure finding host (): no default host found >"
< timeout on docker API check >

 ======Command Output======
 time="2018-04-13T14:57:13-07:00" level=info msg="Docker API endpoint check failed: failed to connect to https://00.00.00.00:2376/info: i/o timeout"
 time="2018-04-13T14:57:13-07:00" level=info msg="Collecting f26795ab-4fcd-478c-89b1-c4205f4f073d vpxd.log"
 time="2018-04-13T14:57:14-07:00" level=info msg="\tAPI may be slow to start - try to connect to API after a few minutes:"
 time="2018-04-13T14:57:14-07:00" level=info msg="\t\tRun command: docker -H 00.00.00.00:2376 --tls info"
 time="2018-04-13T14:57:14-07:00" level=info msg="\t\tIf command succeeds, VCH is started. If command fails, VCH failed to install - see documentation for troubleshooting."
 time="2018-04-13T14:57:14-07:00" level=error msg="vic/cmd/vic-machine/create.(*Create).Run.func3: Create error: context deadline exceeded\nvic/cmd/vic-machine/create.(*Create).Run:741 Create\nvic/cmd/vic-machine/common.NewOperation:27 vic-machine-windows"
 time="2018-04-13T14:57:14-07:00" level=error msg=--------------------
 time="2018-04-13T14:57:14-07:00" level=error msg="vic-machine-windows create failed: Creating VCH exceeded time limit of 3m0s. Please increase the timeout using --timeout to accommodate for a busy vSphere target\n"
 ========End Output========

When connected to the VIC admin browser UI,the port-later shows the following messages:

 ==========00.00.00.00:2378/logs/tail/port-layer.log==========
 Apr 13 2018 21:42:44.091Z FATAL configure_port_layer ERROR: Failure finding host (): no default host found
 time="2018-04-13T21:42:47Z" level=info msg="Launching portlayer server pprof server on 127.0.0.1:6063" 
 …
 …
 Apr 13 2018 21:54:21.261Z FATAL configure_port_layer ERROR: Failure finding host (): no default host found
 time="2018-04-13T21:54:25Z" level=info msg="Launching portlayer server pprof server on 127.0.0.1:6063"
 time="2018-04-13T21:54:25Z" level=info msg="{Formatter:0xc42032fb20 Level:debug Syslog:<nil>}"

On checking the Port-Layer logs you see entries similar to : 

 ==========/var/log/vic/port-layer.log==========
 Apr 13 2018 21:54:25.677Z DEBUG GuestInfoSource: key: guestinfo.vice./vic_machine_create_options~, value: "--target=https://it-vcntr-app-01.xx.xx.xx/Campus|--name=IT000000_7000_IT-VMVCH-APP-01|--compute-resource=SE-ABC-AB-BC-03|--image-store=00_000_00|--dns-server=00.0.00.0|--dns-server=00.0.00.00|--bridge-network=VM-1234-SE-VCH|--public-network=VM-1234-VMware|--public-network-gateway=00.00.000.0|--public-network-ip=00.00.000.00|--container-network=VM-1234-Systems-Management:VM-1234-SYSMGNT|--tls-cname=it-ABCD-app-01.xx.xx.xx|--no-tlsverify=true|--debug=1", error: %!s(<nil>)
 Apr 13 2018 21:54:25.677Z DEBUG op=306.1: Creating VMOMI session with thumbprint 2B:91:76:01:7B:C4:15:B9:73:AB:6A:62:90:F5:99:9B:EC:31:EA:1F
 Apr 13 2018 21:54:25.978Z DEBUG Session Environment Info:
 Apr 13 2018 21:54:25.978Z DEBUG op=306.1: vSphere resource cache populating...
 Apr 13 2018 21:54:26.047Z DEBUG op=306.1: Cached dc: Campus
 Apr 13 2018 21:54:26.177Z DEBUG op=306.1: Cached cluster: /Campus/host/TBE/vPod2/AB-BC-CD-01
 Apr 13 2018 21:54:26.707Z DEBUG op=306.1: Cached ds: 02_948_30
 Apr 13 2018 21:54:26.726Z DEBUG op=306.1: Cached pool: /Campus/host/TBE/vPod2/AB-BC-CD-01/Test
 Apr 13 2018 21:54:26.734Z DEBUG op=306.1: Cached folders: Campus
 Apr 13 2018 21:54:26.736Z DEBUG op=306.1: Error count populating vSphere cache: (1)
 
 Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere Integrated Containers 1.x

Cause

This issue occurs when ESXI Cluster is nested within folders.
 

Resolution

This is a known issue affecting vSphere Integrated Containers if deployed in Nested Environments. There is currently no resolution. This issue is to be addressed in future versions of the product.

Additional Information

For updates regarding this issue subscribe to this KB and look for the release notes for future release versions.