PKS Container creation fails due to HyperBus interface vmk50 not being present on ESXI. PKS 1.2 NSX-T 2.1
search cancel

PKS Container creation fails due to HyperBus interface vmk50 not being present on ESXI. PKS 1.2 NSX-T 2.1

book

Article ID: 330616

calendar_today

Updated On:

Products

VMware NSX VMware PKS 1.x

Issue/Introduction

Purpose is to overcome container creation failure due to HyperBus interface vmk50 not being present on ESXI. 

Symptoms:
Container creation fails. 

Log snippet for reference.

2018-12-05T20:28:46.137Z 37f52426-####-####-####-##########19 NSX 11471 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] oslo.
privsep.daemon privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/CAP_
DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/none
1 2018-12-05T20:28:46.137Z 37f52426-####-####-####-##########19 NSX 11471 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] oslo.privsep.daemon privsep daemon running as pid 11471
1 2018-12-05T20:28:46.256Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Created veth pair nsx_agent_outer, nsx_agent_inner
1 2018-12-05T20:28:46.465Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Configured interface nsx_agent_inner with IP 169.254.1.0/31, MAC 00:50:56:##:##:##
1 2018-12-05T20:28:46.480Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.agent Added OVS port nsx_agent_outer with vlan 4094
1 2018-12-05T20:28:46.520Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.nsxrpc_client Listening NSX RPC connection...
1 2018-12-05T20:28:46.521Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] cli.server.container_cli_server Starting node_agent CLI server
1 2018-12-05T20:28:46.532Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher CNI socket is listening...
1 2018-12-05T20:39:05.068Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be configured
1 2018-12-05T20:39:06.073Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher Failed to find OVS port for container 3497dd0a78a264bf97e4de52a88ce6e57158db92b92107f0abcdfce9########: __init__() takes exactly 1 argument (2 given)
1 2018-12-05T20:44:05.328Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be configured
1 2018-12-05T20:44:06.331Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="INFO"] nsx_ujo.agent.cni_watcher Failed to find OVS port for container f2226c12e3786a35e180f669159e1c48dbf44981c7c558b78b8f25da########: __init__() takes exactly 1 argument (2 given)
1 2018-12-05T20:49:05.606Z 37f52426-####-####-####-##########19 NSX 11443 - [nsx@6876 comp="nsx-container-node" subcomp="nsx_node_agent" level="ERROR" errorCode="NCP01004"] nsx_ujo.agent.cni_watcher Unable to retrieve network info for container nsx.kube-system.kube-dns-6b697fcdbd-fb4tg, network interface for it will not be conf

Environment

VMware PKS 1.x

VMware NSX-T 2.3

Cause

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.

Resolution

Upgrade to NSX-T 2.3 and post that removed and added back the Fabric nodes or use the below workaround.

Issue 1998217: HyperBus interface vmk50 might be missing on vSphere ESXi causing container creation failure

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.
 

Workaround: Complete the following steps.

  1. Retrieve the vmk50 port ID using CLI on vSphere ESXi
    net-dvs | grep vmk50 -C 10
  2. Create the vmk50 interface on vSphere ESXi.
    esxcli network ip interface add -P <port-id from step-1> -s DvsPortset-0 -i vmk50 -N hyperbus
  3. Assign an IP address to the vmk50 interface.
    esxcfg-vmknic -i 169.254.1.1 -n 255.255.0.0 -s DvsPortset-0 -v <port-id from step-1> -N hyperbus


Workaround:
Issue 1998217: HyperBus interface vmk50 might be missing on vSphere ESXi causing container creation failure

Container is not created because the HyperBus interface vmk50 might be missing on vSphere ESXi.
 

Workaround: Complete the following steps.

  1. Retrieve the vmk50 port ID using CLI on vSphere ESXi
    net-dvs | grep vmk50 -C 10
  2. Create the vmk50 interface on vSphere ESXi.
    esxcli network ip interface add -P <port-id from step-1> -s DvsPortset-0 -i vmk50 -N hyperbus
  3. Assign an IP address to the vmk50 interface.
    esxcfg-vmknic -i 169.254.1.1 -n 255.255.0.0 -s DvsPortset-0 -v <port-id from step-1> -N hyperbus


Additional Information

Impact/Risks:
None

Powered by Wolken Software