To identify a known issue where the HCX Transport Analytics feature may report a lower Available Bandwidth then is actually available between connected sites
Symptoms:
Starting with HCX Manager software 4.8.0, the Transport Analytics feature may report a lower Available Bandwidth then is actually available between connected sites
HCX Manager UI >> Transport Analytics >> Uplinks Overview > RUN TEST The HCX Manager app.log found in /common/logs/admin will show the HCX appliance selected for the Transport Analytics on-demand test and the result of that test. The test result in app.log will match that shown in the HCX Manager Transport Analytics UI
In the example below the HCX Appliance selected for the on-demand test is NE-I6 and the Available Bandwidth reported by the test is "downloadKbps":130181,"uploadKbps":102441
2024-02-26 21:21:57.774 UTC [InterconnectService_SvcThread-51105, IX:########-####-####-####-##########43, J:b85a1b25, , TxId: ########-####-####-####-##########b7] INFO c.v.h.i.u.InterconnectApplianceUtil- getIP : applianceName SM_localxxxxx-14bc6-NE-I6 networkId network-########-####-####-####-##########9f IP 172.xx.xx.11 2024-02-26 21:21:59.269 UTC [InterconnectService_SvcThread-51140, IX:########-####-####-####-##########43, J:b85a1b25, , TxId: ########-####-####-####-##########b7] INFO c.v.h.i.u.InterconnectUnderlayUplinkUtil- OnDemand test Url : https://172.xx.xx.202:9443/uc/ondemand/start requestObject {"localAddress":"172.xx.xx.11","remoteAddress":"172.xx.xx.12"} responseBody : {"measureTime":1708982481,"status":2,"error":"","localAddress":"172.xx.xx.11","remoteAddress":"172.xx.xx2.12","downloadKbps":0,"uploadKbps":0,"lossRate":0,"latency":"","txPmtu":0,"rxPmtu":0} 2024-02-26 21:23:40.374 UTC [InterconnectService_SvcThread-51147, IX:########-####-####-####-##########43, J:b85a1b25, , TxId: ########-####-####-####-##########b7] INFO c.v.h.i.u.InterconnectUnderlayUplinkUtil- OnDemand test result Url : https://172.xx.xx.202:9443/uc/ondemand/results requestObject {"localAddress":"172.xx.xx.11","remoteAddress":"172.xx.xx.12"} responseBody : {"measureTime":1708982481,"status":0,"error":"","localAddress":"172.xx.xx.11","remoteAddress":"172.xx.xx.12","downloadKbps":130181,"uploadKbps":102441,"lossRate":0,"latency":"2.930\/4.898\/48.518","txPmtu":1400,"rxPmtu":1400}
When manually running the performance test "perftest fou" on the same HCX appliance NE-I6 however, an accurate and larger value is reported for available bandwidth when compared to the values reported by the Transport Analytics UI
In the below output a 468Mbps downlink and 436Mbps uplink available bandwidth are reported by "perftest fou"
[admin@host:SM_localxxxxx-NE-I6] perftest fou ++++++++++ StartTest ++++++++++ ---------- Fou-0 [te_0, 192.168.2.63 >>> 192.168.2.72] ---------- Connecting to host 192.168.2.72, port 5201 [ 5] local 192.168.2.63 port 45703 connected to 192.168.2.72 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-30.00 sec 1.64 GBytes 470 Mbits/sec 1157 1.53 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 1.64 GBytes 470 Mbits/sec 1157 sender [ 5] 0.00-30.02 sec 1.64 GBytes 468 Mbits/sec receiver iperf Done. ++++++++++ StopTest ++++++++++ ++++++++++ StartTest ++++++++++ ---------- Fou-0 [te_0, 192.168.2.63 <<< 192.168.2.72] ---------- Connecting to host 192.168.2.63, port 5201 [ 5] local 192.168.2.72 port 44645 connected to 192.168.2.63 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-30.00 sec 1.53 GBytes 438 Mbits/sec 56 1.18 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 1.53 GBytes 438 Mbits/sec 56 sender [ 5] 0.00-30.08 sec 1.53 GBytes 436 Mbits/sec receiver iperf Done. ++++++++++ StopTest ++++++++++
The issue will be resolved in HCX Manager 4.9.0 software release or higher
Workaround:
On the same HCX appliance used by Transport Analytics (or any other HCX IX/NE appliance associated with the same Connected Sites) manually run the performance test "perftest fou" to determine the available bandwidth between the sites
Check the receiver Bitrate in both directions. In the example below it shows 468Mbps downlink >>> and 436Mbps uplink <<<
ssh as admin into the HCX Manager ccli list go <HCX_appliance_ID#> perftest fou
Example:
admin@host [~]$ ccli Welcome to HCX Central CLI [admin@host] list |----------------------------------------------------------------------------| | Node | Id | Address | State | Selected | |----------------------------------------------------------------------------| | SM_localxxxxx-14bc6-IX-I1 | 3 | 172.xx.xx.204:9443 | Connected | | |----------------------------------------------------------------------------| <snip> |----------------------------------------------------------------------------| | SM_localxxxxx-14bc6-NE-I6 | 1 | 172.xx.xx.202:9443 | Connected | | ------> here !!!! |----------------------------------------------------------------------------| | SM_localxxxxx-14bc6-NE-I7 | 2 | 172.xx.xx.206:9443 | Connected | | |----------------------------------------------------------------------------| <snip> [admin@host] go 1 Switched to node 1. [admin@host:SM_localxxxxx-14bc6-NE-I6] [admin@host:SM_localxxxxx-14bc6-NE-I6] perftest fou ++++++++++ StartTest ++++++++++ ---------- Fou-0 [te_0, 192.168.2.63 >>> 192.168.2.72] ---------- Connecting to host 192.168.2.72, port 5201 [ 5] local 192.168.2.63 port 45703 connected to 192.168.2.72 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-30.00 sec 1.64 GBytes 470 Mbits/sec 1157 1.53 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 1.64 GBytes 470 Mbits/sec 1157 sender [ 5] 0.00-30.02 sec 1.64 GBytes 468 Mbits/sec receiver iperf Done. ++++++++++ StopTest ++++++++++ ++++++++++ StartTest ++++++++++ ---------- Fou-0 [te_0, 192.168.2.63 <<< 192.168.2.72] ---------- Connecting to host 192.168.2.63, port 5201 [ 5] local 192.168.2.72 port 44645 connected to 192.168.2.63 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-30.00 sec 1.53 GBytes 438 Mbits/sec 56 1.18 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-30.00 sec 1.53 GBytes 438 Mbits/sec 56 sender [ 5] 0.00-30.08 sec 1.53 GBytes 436 Mbits/sec receiver iperf Done. ++++++++++ StopTest ++++++++++