[HCX] How to check HCX appliances status in CLI.
search cancel

[HCX] How to check HCX appliances status in CLI.

book

Article ID: 325021

calendar_today

Updated On:

Products

VMware HCX VMware Cloud on AWS

Issue/Introduction

Purpose
If your HCX environment has issues, you can find out the status details in CLI.
This article explains how to do an HCX health check.

Impact
This is for VMware HCX that are at version R133 or newer.


Resolution

Please run these steps on your on-prem side.

Step1. Enabling SSH on the VMware HCX Manager


Step2. Logging In to the VMware HCX Manager CLI

Step3. Using Central CLI to check HCX appliances status

3-1. Type "ccli" to enable CCLI on VMware HCX Manager

3-2. Type "list" to view a list of VMware HCX nodes (appliances). 
       You can find out that any appliances is not selected yet.

3-3. Type "go #" where # is the node ID. 
       As an example of this KB, "0" is HCX-IX、"1" is HCX-NET-EXT.
       Type "go 0" first.

3-4. Type "list" to view a list of VMware HCX nodes.
       You can find out that HCX-IX is selected now.

3-5. Type "hc -d" to view status of VMware HCX-IX node as below.
       You can find out the status of tunnels and service resources and etc.
Sample for HCX-IX:

[root@hcx_ent_manager:SM1-IX-I1] hc -d
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
              Probe HealthChecking
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
(vcenter) : <IP>(443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)

(lookupservice) : <IP>(7444)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)

(SM1-IX-I1-xx-xx-xx-xx-xxxx) : <IP>(9443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)
  |-- gateway status(success)
        |-- Appliance System Status: good
        |-- Peer Site Connectivity: up
                |-- WANOPT 198.xx.xx.xx Status: up
                |-- Tunnel t_0 Status: up       , rx           260266, tx           326547
                |-- Tunnel t_2 Status: up       , rx           456333, tx           321169
                |-- Tunnel t_1 Status: up       , rx           260480, tx           330410
                |-- Tunnel te_0 Status: up      , rx          1105305, tx          1106564

(SM1-NE-I1-xx-xx-xx-xx-xxxx) : <IP>(9443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)
  |-- gateway status(success)
        |-- Appliance System Status: good
        |-- Peer Site Connectivity: up
                |-- Tunnel t_1 Status: up       , rx           261355, tx           260254
                |-- Tunnel t_2 Status: up       , rx           260805, tx           261243
                |-- Tunnel t_0 Status: up       , rx           260760, tx           261336
                |-- Tunnel t_3 Status: up       , rx           260734, tx           260780
                |-- Tunnel t_4 Status: up       , rx           260758, tx           260806
                |-- Tunnel te_0 Status: up      , rx          1432860, tx          1564406

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
            Node Units HealthChecking
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
[+] systemResource:
        [-] cpu
                [*] cpu.idle             ............ good
                [*] cpu.loadAvg          ............ good
                   > CPU load average high (1min: 0.090000, 5min: 0.050000, 15min: 0.010000), threshold: 6.400000
        [-] memory
                [*] memory.free          ............ good
                   > Free memory (free: 2555580416 bytes, total: 3144695808 bytes)
        [-] disk
                [*] disk.root.free       ............ good
                   > Free root partition (free: 335876096 bytes, 27% of total)
                [*] disk.log.free        ............ good
                   > Free log partition (free: 510021632 bytes, 92% of total)
                [*] disk.coredump.free   ............ good
                   > Free coredump partition (free: 395269120 bytes, 94% of total)

[+] systemService:
        [-] service.status
                [*] cgw                  ............ good
                   > Service cgw is running properly.
                [*] hbrsrv               ............ good
                   > Service hbrsrv is running properly.
                [*] lwdproxy             ............ good
                   > Service lwdproxy is running properly.
                [*] ntpd                 ............ good
                   > Service ntpd is running properly.
                [*] fcron                ............ good
                   > Service fcron is running properly.
                [*] vmtoolsd             ............ good
                   > Service vmtoolsd is running properly.
                [*] watchdog             ............ good
                   > Service watchdog is running properly.
                [*] hamon                ............ good
                   > Service hamon is running properly.
                [*] ipsec                ............ good
                   > Service ipsec is running properly.

[+] encryptService:
        [-] encrypt.daemons
                [*] ipsec                ............ good
                   > Service ipsec is running properly.
        [-] encrypt.tunnels
                [*] encrypt.configstate  ............ good
                [*] encrypt.tunnelstate  ............ good
                [*] encrypt.tunnelconn.t_0 ............ good
                   > Tunnel t_0 connectivity is UP.
                [*] encrypt.tunnelconn.t_1 ............ good
                   > Tunnel t_1 connectivity is UP.
                [*] encrypt.tunnelconn.t_2 ............ good
                   > Tunnel t_2 connectivity is UP.

[+] fouService:
        [-] fou.tunnels
                [*] fou.tunnelstate.ipip_te_0 ............ good
                [*] fou.flow.{ <IP>:4500 --> <IP>:4500, state:  } { <IP>:4500 --> <IP>:4500, state: [ASSURED] } ............ good
                [*] fou.configstate      ............ good
                [*] fou.tunnelstate      ............ good
3-6. Type "go 1" second.

3-7. Type "list" to view a list of VMware HCX nodes.
       You can find out that HCX-NET-EXT is selected now.

3-8. Type "hc -d" to view status of VMware HCX-NET-EXT node as below.
       You can find out the status of tunnels and service resources and etc for HCX-NET-EXT as well.
Sample for HCX-NET-EXT:

[root@hcx_ent_manager:SM1-NE-I1] hc -d
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
              Probe HealthChecking
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
(vcenter) : <IP>(443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)

(lookupservice) : <IP>(7444)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)

(SM1-IX-I1-xx-xx-xx-xx-xxxx) : <IP>(9443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)
  |-- gateway status(success)
        |-- Appliance System Status: good
        |-- Peer Site Connectivity: up
                |-- WANOPT 198.xx.xx.xx Status: up
                |-- Tunnel t_0 Status: up       , rx           260128, tx           326375
                |-- Tunnel t_1 Status: up       , rx           260342, tx           330231
                |-- Tunnel te_0 Status: up      , rx          1104686, tx          1105945
                |-- Tunnel t_2 Status: up       , rx           456089, tx           321000


(SM1-NE-I1-xx-xx-xx-xx-xxxx) : <IP>(9443)
  |-- icmp(success)
  |-- ssl connection(success)
  |-- ssl handshake(success)
  |-- gateway status(success)
        |-- Appliance System Status: good
        |-- Peer Site Connectivity: up
                |-- Tunnel t_0 Status: up       , rx           260617, tx           261192
                |-- Tunnel t_2 Status: up       , rx           260661, tx           261100
                |-- Tunnel t_3 Status: up       , rx           260591, tx           260636
                |-- Tunnel te_0 Status: up      , rx          1432043, tx          1563512
                |-- Tunnel t_1 Status: up       , rx           261211, tx           260111
                |-- Tunnel t_4 Status: up       , rx           260614, tx           260662

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
            Node Units HealthChecking
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
[+] systemResource:
        [-] cpu
                [*] cpu.idle             ............ good
                [*] cpu.loadAvg          ............ good
                   > CPU load average high (1min: 0.000000, 5min: 0.000000, 15min: 0.000000), threshold: 6.400000
        [-] memory
                [*] memory.free          ............ good
                   > Free memory (free: 2563272704 bytes, total: 3144695808 bytes)
        [-] disk
                [*] disk.root.free       ............ good
                   > Free root partition (free: 352788480 bytes, 29% of total)
                [*] disk.log.free        ............ good
                   > Free log partition (free: 493477888 bytes, 89% of total)
                [*] disk.coredump.free   ............ good
                   > Free coredump partition (free: 372967424 bytes, 88% of total)

[+] systemService:
        [-] service.status
                [*] cgw                  ............ good
                   > Service cgw is running properly.
                [*] hbrsrv               ............ good
                   > Service hbrsrv is running properly.
                [*] lwdproxy             ............ good
                   > Service lwdproxy is running properly.
                [*] ntpd                 ............ good
                   > Service ntpd is running properly.
                [*] fcron                ............ good
                   > Service fcron is running properly.
                [*] vmtoolsd             ............ good
                   > Service vmtoolsd is running properly.
                [*] watchdog             ............ good
                   > Service watchdog is running properly.
                [*] hamon                ............ good
                   > Service hamon is running properly.
                [*] ipsec                ............ good
                   > Service ipsec is running properly.

[+] encryptService:
        [-] encrypt.daemons
                [*] ipsec                ............ good
                   > Service ipsec is running properly.
        [-] encrypt.tunnels
                [*] encrypt.configstate  ............ good
                [*] encrypt.tunnelstate  ............ good
                [*] encrypt.tunnelconn.t_0 ............ good
                   > Tunnel t_0 connectivity is UP.
                [*] encrypt.tunnelconn.t_1 ............ good
                   > Tunnel t_1 connectivity is UP.
                [*] encrypt.tunnelconn.t_2 ............ good
                   > Tunnel t_2 connectivity is UP.
                [*] encrypt.tunnelconn.t_3 ............ good
                   > Tunnel t_3 connectivity is UP.
                [*] encrypt.tunnelconn.t_4 ............ good
                   > Tunnel t_4 connectivity is UP.

[+] fouService:
        [-] fou.tunnels
                [*] fou.tunnelstate.ipip_te_0 ............ good
                [*] fou.flow.{ <IP>:4500 --> <IP>:4500, state:  } { <IP>:4500 --> <IP>:4500, state: [ASSURED] } ............ good
                [*] fou.configstate      ............ good
                [*] fou.tunnelstate      ............ good

Additional Information

VMware HCX Troubleshooting