Troubleshooting Tools for Networking on vCenter Server Appliance 6.5 and Above
search cancel

Troubleshooting Tools for Networking on vCenter Server Appliance 6.5 and Above

book

Article ID: 322847

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

vCenter Server Appliance 6.5 and above is built using VMware Photon OS. Previous versions of the vCenter Server Appliance were built using Suse Linux Enterprise Server (SLES).

This article explains some of the basic commands that may be useful when troubleshooting network related issues for Photon OS for vCenter Server Appliance 6.5 and above.

Important: This article is explicitly for vCenter Server Appliance 6.5 and above and may not work for previous versions.


Environment

VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.5.x

Resolution

Use of commands IP and SS instead of IPconfig and netstat.
 
Although the ifconfig command and the netstat command work on Photon OS, VMware recommends that you use the ip or ss commands. The ifconfig and netstat commands are deprecated.
 
Photon OS Command SLES Command
ip addr ifconfig -a
ss netstat
ip route route
ip maddr netstat -g
ip link set eth0 up ifconfig eth0 up
ip -s neigh arp -v
ip link set eth0 mtu 9000 ifconfig eth0 mtu 9000
Verifying the Status of Network up-link using the command:
 
  • The command networkctl shows information about network connections that helps you configure networking services and troubleshoot networking problems.

    For example:

    # networkctl
    IDX LINK TYPE OPERATIONAL SETUP
    1 lo loopback carrier unmanaged
    2 eth0 ether routable configured


    Two up-links listed.
     
  • The command networkctl statusdisplays information about the network up-link status

    For example:

    # networkctl status
    State: routable
    Address: 10.x.x.x on eth0
    Gateway: 10.x.x.x (Cisco Systems, Inc) on eth0
    DNS: 10.x.x.x
    10.x.x.x
    Domain: xyz.com

     
  • To display the up-link configuration, use the command networkctl status etho

    For example:

    # networkctl status eth0

    You see output similar to:

    2: eth0
    Link File: /usr/lib/systemd/network/99-default.link
    Network File: /etc/systemd/network/10-eth0.network
    Type: ether
    State: routable (configured)
    Path: pci-0000:03:00.0
    Driver: vmxnet3
    Vendor: VMware
    Model: VMXNET3 Ethernet Controller
    HW Address: 00:0c:29:9f:c9:09 (VMware, Inc.)
    MTU: 1500
    Address: 10.x.x.x
    Gateway: 10.x.x.x (Cisco Systems, Inc)
    DNS: 10.x.x.x
    10.x.x.x
    Domain: xyz.com
    CLIENTID: 01000c299fc909
     
Using the command TCPDUMP:
 
The Tcpdump Commands captures and analyzes packets on the network interface. Tcpdump is available by default on the vCenter Sever Appliance 6.5

When running the #tcpdump --help command, you see output similar to:

tcpdump version 4.7.4
libpcap version 1.7.4
Usage: tcpdump [-aAbdDefhHIJKlLnNOpqRStuUvxX#] [ -B size ] [ -c count ]
[ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G seconds ]
[ -i interface ] [ -j tstamptype ] [ -M secret ] [ --number ]
[ -Q in|out|inout ]
[ -r file ] [ -s snaplen ] [ --time-stamp-precision precision ]
[ --immediate-mode ] [ -T type ] [ --version ] [ -V file ]
[ -w file ] [ -W filecount ] [ -y datalinktype ] [ -z command ]
[ -Z user ] [ expression ]

 
Using the command NETCAT:

 

The Netcat can be used to test port connectivity.
 
When running the # netcat -h command, you see output similar to:
 
GNU netcat 0.7.1, a rewrite of the famous networking tool.
Basic usages:
connect to somewhere: netcat [options] hostname port [port] ...
listen for inbound: netcat -l -p port [options] [hostname] [port] ...
tunnel to somewhere: netcat -L hostname:port -p port [options]
 
Mandatory arguments to long options are mandatory for short options too.
Options:
-c, --close close connection on EOF from stdin
-e, --exec=PROGRAM program to exec after connect
-g, --gateway=LIST source-routing hop point[s], up to 8
-G, --pointer=NUM source-routing pointer: 4, 8, 12, ...
-h, --help display this help and exit
-i, --interval=SECS delay interval for lines sent, ports scanned
-l, --listen listen mode, for inbound connects
-L, --tunnel=ADDRESS:PORT forward local port to remote address
-n, --dont-resolve numeric-only IP addresses, no DNS
-o, --output=FILE output hexdump traffic to FILE (implies -x)
-p, --local-port=NUM local port number
-r, --randomize randomize local and remote ports
-s, --source=ADDRESS local source address (ip or hostname)
-t, --tcp TCP mode (default)
-T, --telnet answer using TELNET negotiation
-u, --udp UDP mode
-v, --verbose verbose (use twice to be more verbose)
-V, --version output version information and exit
-x, --hexdump hexdump incoming and outgoing traffic
-w, --wait=SECS timeout for connects and final net reads
-z, --zero zero-I/O mode (used for scanning)
Remote port number can also be specified as range. Example: '1-1024'
 
 
Using the command LSOF:

To lsof command and can be used to view a list of running processes and the ports they are consuming.
When running to # lsof -i -P -n command, you see output similar to:
 
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd 1 root 83u IPv6 11427 0t0 UDP *:69
ntpd 909 ntp 16u IPv6 11786 0t0 UDP *:123
ntpd 909 ntp 17u IPv4 11789 0t0 UDP *:123
ntpd 909 ntp 18u IPv4 11793 0t0 UDP 127.0.0.1:123
ntpd 909 ntp 19u IPv6 11795 0t0 UDP [::1]:123
ntpd 909 ntp 21u IPv4 26108 0t0 UDP 10.x.x.x:123
systemd-n 4146 systemd-network 13u IPv4 25130 0t0 UDP 10.x.x.x:68
rpcbind 4375 rpc 6u IPv4 27963 0t0 UDP *:111
rpcbind 4375 rpc 7u IPv4 27964 0t0 UDP *:731
rpcbind 4375 rpc 8u IPv4 27965 0t0 TCP *:111 (LISTEN)
rpcbind 4375 rpc 9u IPv6 27966 0t0 UDP *:111
rpcbind 4375 rpc 10u IPv6 27967 0t0 UDP *:731
rpcbind 4375 rpc 11u IPv6 27968 0t0 TCP *:111 (LISTEN)
dcerpcd 4609 root 3u IPv4 28201 0t0 TCP *:135 (LISTEN)
dcerpcd 4609 root 6u IPv4 28205 0t0 UDP *:135
dcerpcd 4609 root 7u IPv4 95629 0t0 TCP 10.x.x.x:39766->10.x.x.x:2020 (ESTABLISHED)
dcerpcd 4609 root 8u IPv4 95631 0t0 TCP 10.x.x.x:36094->10.x.x.x:2012 (ESTABLISHED)
dcerpcd 4609 root 9u IPv4 96413 0t0 TCP 10.x.x.x:43905->10.x.x.x:2015 (ESTABLISHED)
vmafdd 4616 root 3u IPv4 95630 0t0 TCP 10.x.x.x:2020->10.x.x.x:39766 (ESTABLISHED)
vmafdd 4616 root 21u IPv4 28229 0t0 TCP *:2020 (LISTEN)
vmcad 4862 root 19u IPv4 28592 0t0 TCP *:2014 (LISTEN)
vmdird 5193 root 22u IPv4 29202 0t0 TCP *:2012 (LISTEN)
vmdird 5193 root 25u IPv4 86892 0t0 TCP 10.x.x.x:389->10.x.x.x:42080 (ESTABLISHED)
vmdird 5193 root 26u IPv4 31970 0t0 TCP *:389 (LISTEN)
vmdird 5193 root 27u IPv4 31971 0t0 TCP *:11711 (LISTEN)
vmdird 5193 root 28u IPv4 29268 0t0 TCP *:636 (LISTEN)
vmdird 5193 root 29u IPv4 29269 0t0 TCP *:11712 (LISTEN)
vmdird 5193 root 32u IPv4 31973 0t0 TCP *:88 (LISTEN)
vmdird 5193 root 33u IPv4 31975 0t0 UDP *:88
vmdird 5193 root 34u IPv4 95632 0t0 TCP 10.x.x.x:2012->10.x.x.x:36094 (ESTABLISHED)
vmdird 5193 root 35u IPv4 91988 0t0 TCP 10.x.x.x:389->10.x.x.x:42456 (ESTABLISHED)
 
Using the command systemd-networkd:

You can check its status of the network service by running the # systemctl status systemd-networkd command:

You see output similar to:

# systemctl status systemd-networkd
systemd-networkd.service - Network Service
Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2016-09-28 09:23:08 UTC; 9min ago
Docs: man:systemd-networkd.service(8)
Main PID: 4146 (systemd-network)
Status: "Processing requests..."
Tasks: 1
Memory: 276.0K
CPU: 10ms
CGroup: /system.slice/systemd-networkd.service
4146 /lib/systemd/systemd-networkd
 
09:23:08 localhost.localdom systemd[1]: Starting Network Service...
09:23:08 localhost.localdom systemd-networkd[4146]: Enumeration completed
09:23:08 localhost.localdom systemd[1]: Started Network Service.
09:23:08 localhost.localdom systemd-networkd[4146]: eth0: Gained carrier
09:23:11 localhost.localdom systemd-networkd[4146]: eth0: DHCPv4 address 10.x.x.x/20 via 10.x.x.x
09:23:11 localhost.localdom systemd-networkd[4146]: eth0: Configured

The network configuration of the eth0 interface is stored in the /etc/systemd/network/10-eth0.network file

Run the # cat /etc/systemd/network/10-eth0.network command to display the content of the file.

You see output similar to:

# cat /etc/systemd/network/10-eth0.network

[Match]
Name=eth0
[Network]
Gateway=10.x.x.x
Address=10.x.x.x/x
DHCP=no
[DHCP]
UseDNS=false



Additional Information