Interpreting NSX Edge Interface stats
search cancel

Interpreting NSX Edge Interface stats

book

Article ID: 324167

calendar_today

Updated On:

Products

VMware NSX Networking

Issue/Introduction

The NSX Edge cli provides detailed stats for Gateway interfaces. 
This KB provides details on how to interpret these stats and recommended actions that can be taken.

Example of Edge cli
 
Edge1> get logical-router interface 8638183a-7126-4401-b8cb-175bfce4664e stats
Wed Jan 24 2024 UTC 07:09:38.729
interface   : 8638183a-7126-4401-b8cb-175bfce4664e
ifuid       : 298
VRF         : 5dfb911e-7c35-4c8d-b42c-12323ade5fdb
name        : tier0-interface-90-90-90-2
IP/Mask     : 90.90.90.2/24;2010::250:56ff:fea6:cb7c/64(A);2010::1/64(A);fe80::250:56ff:fea6:cb7c/64(A)
MAC         : 00:50:56:a6:cb:7c
VLAN        : 8
LS port     : 7ea2eb90-b82b-4f74-9349-ac9b2244af9f
urpf-mode   : STRICT_MODE
admin       : up
op_state    : up
MTU         : 1500
statistics
    RX-Packets  : 275749
    RX-Bytes    : 23938078
    RX-Drops    : 4996
        Blocked     : 0
        DST-Unsupported: 4515
        Firewall    : 0
        Malformed   : 0
        No-Receiver : 0
        No-Route    : 0
        RPF-Check   : 81
        Protocol-Unsupported: 396
        IPv6        : 4
        Port-Unsupported: 0
        TTL-Exceeded: 0
        Kni         : 0
        Rate-Limit  : 0
        IPsec       : 0
        IPsec-NoSA  : 0
        IPsec-NoVTI : 0
    TX-Packets  : 203743
    TX-Bytes    : 17505975
    TX-Drops    : 0
        Blocked     : 0
        Firewall    : 0
        Frag-Needed : 0
        No-neighbor : 0
        No-Memory   : 0
        No-Linked-Port: 0
        Rate-Limit  : 0
        IPsec       : 0
        IPsec-NoSA  : 0
        IPsec-NoVTI : 0
        IPsec-Policy-Error: 0
        IPsec-Policy-Block: 0
    IP Ressemble
        Fragments-OK: 0
        Fragemnts-Error: 0
        Fragments-Timeout: 0
    IP Fragment
        Fragments-OK: 0
        Fragments-Error: 0


Environment

VMware NSX 4.1.0

Resolution


RX Interface Stats
 
Stats (Rx)DescriptionAction
BlockedPacket dropped due to interface being blocked/admin-down due to SR of the edge node is in standby state

Some packets dropped due to blocked state is expected during failover/fallback case. 

if this counter continue to increase, the following command should be used to check admin and internal_operation:

nsx> get logical-router interface c61a8048-21bf-498a-88e2-4a4a145d9244 | json
{
  "admin""up",
  "internal_operation""up",
}
DST-UnsupportedPacket dropped due to unsupported destination like mcast (but mcast is not enabled), loopback or reserved address

Start capture command can be used to find out the unsupported destination address and where is the traffic coming from (source address)

start capture interface c61a8048-21bf-498a-88e2-4a4a145d9244
FirewallPacket dropped due to firewallCheck if there is drop rule or state mismatch traffic
Edge1> get firewall b7d57745-8604-4d3c-b93d-f63164c8752b ruleset type firewall rules
Mon Feb 12 2024 UTC 03:43:32.837
Rule count: 1
    Rule ID   : 1005
    Rule      : inout protocol any stateless from any to any accept
 
Edge1> get firewall b7d57745-8604-4d3c-b93d-f63164c8752b ruleset type firewall stats
Mon Feb 12 2024 UTC 03:43:20.547
Rule count: 1
    Rule ID             : 1005
    Input bytes         : 253299828
    Output bytes        : 275434116
    Input packets       : 4189054
    Output packets      : 4607874
    Evaluations         : 8796932
    Hits                : 8796928
    Active connections  : 0
MalformedPacket dropped due to malformed fields, ip checksum or L4 checksum could be the reason of malformed trafficCheck the sender checksum offload configuration
No-ReceiverPacket dropped due to destination being a lrouter with no receiver i.e. proto not supported by it or no linked tunnel existsCheck configuration of lrouter port, if its GRE port, then it should be linked to GRE tunnel port.
root@Edge1:~# edge-appctl -t /var/run/vmware/edge/dpd.ctl lrouter_port/show 2f406d88-2515-4eba-a84b-7d2a1f05195d | json_pp
{
   "ifuuid" "2f406d88-2515-4eba-a84b-7d2a1f05195d",
    ...
   "lrouter" "feca5512-4bb4-4346-819c-fd6a9bcac289",
    ...
   "name" "test-gre - 0",
    ...
   "peer" "37c4f125-84e0-494e-abc5-a941ecf7be8c",     >>>> This should be GRE tunnel port UUID
    ...
   "ptype" "gre-port",
}
 
Edge1> get tunnel-port 37c4f125-84e0-494e-abc5-a941ecf7be8c
Mon Feb 12 2024 UTC 03:29:41.339
Tunnel      : 37c4f125-84e0-494e-abc5-a941ecf7be8c
IFUID       : 394
LOCAL       : 40.40.40.1
REMOTE      : 5.5.5.5
ENCAP       : GRE
GRETAP VRFID: 2
No-RoutePacket dropped due to routing failure or invalid egress port

Check L3 forwarding table for dst IP

nsx> get logical-router 1eef3979-1bfb-4b41-b936-cf3451158d6c forwarding
Fri Jan 26 2024 PST 13:55:51.367
Logical Router
UUID                                   VRF    LR-ID  Name                              Type                      
1eef3979-1bfb-4b41-b936-cf3451158d6c   2      8      SR-T0-PIAS-IDMZ                   SERVICE_ROUTER_TIER0      
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC     
0.0.0.0/0          10.10.138.14                              route       3c71fae6-1a78-4203-92ef-ff996ca7a23f   00:1c:73:00:00:95
                   10.10.138.10                                          59b73791-3234-422e-b83e-0bef7283c0cc   00:1c:73:00:00:95
RPF-CheckPacket dropped due to no reverse path to destination

Check RPF configuration (urpf-mode) and L3 forwarding table for SRC IP
 

Edge1> get logical-router interface 2f406d88-2515-4eba-a84b-7d2a1f05195d
Mon Feb 12 2024 UTC 03:37:45.617
interface   : 2f406d88-2515-4eba-a84b-7d2a1f05195d
ifuid       : 395
VRF         : feca5512-4bb4-4346-819c-fd6a9bcac289
name        : test-gre - 0
mode        : lif
IP/Mask     : 172.16.10.1/24
Fwd-mode    : IPV4_AND_IPV6
MAC         : 02:50:56:01:00:00
LS port     : 37c4f125-84e0-494e-abc5-a941ecf7be8c
urpf-mode   : STRICT_MODE                          >>>>>>>>>>>>>
admin       : up
op_state    : up
MTU         : 1476
arp_proxy   :
 
Edge1> get logical-router feca5512-4bb4-4346-819c-fd6a9bcac289 forwarding
Mon Feb 12 2024 UTC 03:38:14.600
Logical Router
UUID                                   VRF    LR-ID  Name                              Type
feca5512-4bb4-4346-819c-fd6a9bcac289   2      3      SR-tier0                          SERVICE_ROUTER_TIER0
IPv4 Forwarding Table
IP Prefix          Gateway IP                                Type        UUID                                   Gateway MAC
1.1.1.0/25         100.64.1.1                                route       61ca295f-14b7-5c89-a6e0-f4bee9a153be
1.1.2.0/24         100.64.1.1                                route       61ca295f-14b7-5c89-a6e0-f4bee9a153be
1.1.3.0/24                                                   route       e6807a42-0100-4776-b47e-90fbfc5c81ba
1.1.3.1/32                                                   route       73555c8d-033b-5bd7-ae7d-cdc315fdb072
Protocol-UnsupportedPacket dropped due to known protocol like ARP, ICMP, DHCP but cannot be decoded completelyNo action unless there is traffic connection issue, such as no arp, no icmp reply and etc.
TTL-ExceededPacket dropped due to TTL exceedingThere is a L3 loop, check forwarding information of the complete setup
KniPacket dropped due to lrouter port's companion KNI port failed to sendCheck cpu utilization of linux process such as L7 LB, this is due to linux side not able to dequeue the pkt fast enough.
# top
top - 03:18:57 up 8 days,  7:17,  1 user,  load average: 1.712.202.32
Tasks: 238 total,   3 running, 235 sleeping,   0 stopped,   0 zombie
%Cpu(s):  7.9 us, 13.6 sy,  0.0 ni, 77.9 id,  0.1 wa,  0.0 hi,  0.5 si,  0.0 st
KiB Mem :  7842940 total,   136860 free,  5437616 used,  2268464 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  2334760 avail Mem
 
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+    TGID COMMAND
2784801 nsx-sha   20   0    3212   1004    916 S   0.0   0.0   0:00.00 2784801 sleep
2784770 root      20   0    8056   4100   3264 R   1.9   0.1   0:00.39 2784770 top
2784735 root      20   0    5168   4244   3504 S   0.0   0.1   0:00.04 2784735 bash
Rate-LimitPacket dropped due to ingress traffic rate higher than configured QOS (traffic contract)

Check if ingress traffic to the lrouter is much higher than the QOS config.
You can find the QOS configured using the below CLI,

Edge1> get logical-router 558ba208-8ae9-4b74-a379-55d94e3577ed qos-config
Mon Jan 29 2024 UTC 06:16:45.835
Logical Router
UUID           : 558ba208-8ae9-4b74-a379-55d94e3577ed
Direction      : Ingress
Rate (Mbps)    : 1
Burst (bytes)  : 1
IPsecPacket dropped during IPsec input processing

Aggregated Rx drop counter for IPsec related drop, further check IPsec tunnel stats for particular drop reason using following command

"get ipsecvpn tunnel stats <IPsec tunnel/VTI UUID>" 

edge1> get ipsecvpn tunnel stats cec70165-6e00-4ae8-93ef-b0338614a5e3
Interface UID                      : 295
Interface UUID                     : cec70165-6e00-4ae8-93ef-b0338614a5e3
VTI UUID                           : cec70165-6e00-4ae8-93ef-b0338614a5e3
 
Stats
    Rx Pkts                            : 0             Tx Pkts                            : 0
    Rx Bytes                           : 0             Tx Bytes                           : 0
    Rx MSS Adjusted                    : 0             Tx MSS Adjusted                    : 0
    Rx MSS Ignored                     : 0             Tx MSS Ignored                     : 0
    Rx Drops                           : 0             Tx Drops                           : 0
    Rx Drop Crypto Failure             : 0             Tx Drop Crypto Failure             : 0
    Rx Drop Enqueue Failure            : 0             Tx Drop Enqueue Failure            : 0
    Rx Drop State Mismatch             : 0             Tx Drop State Mismatch             : 0
    Rx Drop Malformed                  : 0             Tx Drop Malformed                  : 0
    Rx Drop Proto Not Supported        : 0             Tx Drop Proto Not Supported        : 0
    Rx Drop Replay                     : 0             Tx Drop Seq Rollover               : 0
    Rx Drop Inner Malformed            : 0             Tx Drop Fragmentation Needed       : 0
    Rx Drop Policy Nomatch             : 0             Rekey Request Failure              : 0
    Rx Drop Auth Failure               : 0
    Rx Drop Zero Sequence Number       : 0
 
v6 Stats
    Rx Pkts                            : 3236          Tx Pkts                            : 3184
    Rx Bytes                           : 250276        Tx Bytes                           : 631712
    Rx MSS Adjusted                    : 0             Tx MSS Adjusted                    : 0
    Rx MSS Ignored                     : 0             Tx MSS Ignored                     : 0
    Rx Drops                           : 0             Tx Drops                           : 0
    Rx Drop Crypto Failure             : 0             Tx Drop Crypto Failure             : 0
    Rx Drop Enqueue Failure            : 0             Tx Drop Enqueue Failure            : 0
    Rx Drop State Mismatch             : 0             Tx Drop State Mismatch             : 0
    Rx Drop Malformed                  : 0             Tx Drop Malformed                  : 0
    Rx Drop Proto Not Supported        : 0             Tx Drop Proto Not Supported        : 0
    Rx Drop Replay                     : 0             Tx Drop Seq Rollover               : 0
    Rx Drop Inner Malformed            : 0             Tx Drop Fragmentation Needed       : 0
    Rx Drop Policy Nomatch             : 0             Rekey Request Failure              : 0
    Rx Drop Auth Failure               : 0
    Rx Drop Zero Sequence Number       : 0
----------------------------------------------------------------------------------------------------
IPsec-NoSAPacket dropped due to missing IPsec inbound or ingress security association SA

Check Outbound and Inbound SAs installed by running below commands

"get ipsecvpn sad" or

"get ipsecvpn sad summary"

edge1> get ipsecvpn sad summary
Outbound SAs (6):
----------------------------------------------------------------------------------------------------------------------------
 VRF   Rule ID      Src IP            Dest IP           Src Subnet          Dest Subnet         SPI          NAT   Rem Life
----------------------------------------------------------------------------------------------------------------------------
 2     1408217139   5050::100         2424::101                                                 0xc6a6b127    N    1642 sec
 2     334475315    5050::100         2424::101                                                 0xcbe1de7e    N    1581 sec
 2     536870913    1111::10          1111::20          192.168.1.0/24      192.168.5.0/24      0x2e733800    N     922 sec
 5     536870927    192.168.51.100    20.20.20.101      192.168.2.0/24      192.168.7.0/24      0xce11f2c1    N    2279 sec
 2     1610612738   1111::10          1111::20          fec0:1::/64         fec0:5::/64         0x2786ae00    N     880 sec
 5     1610612752   192.168.51.100    20.20.20.101      fec0:2::/64         fec0:7::/64         0xc02800f4    N    2279 sec
 
Inbound SAs (6):
----------------------------------------------------------------------------------------------------------------------------
 VRF   Rule ID      Src IP            Dest IP           Src Subnet          Dest Subnet         SPI          NAT   Rem Life
----------------------------------------------------------------------------------------------------------------------------
 5     2684354575   20.20.20.101      192.168.51.100    192.168.7.0/24      192.168.2.0/24      0xbf806100    N    2279 sec
 2     3555700787   2424::101         5050::100                                                 0xc3061f00    N    1642 sec
 5     3758096400   20.20.20.101      192.168.51.100    fec0:7::/64         fec0:2::/64         0xef28dd00    N    2279 sec
 2     2684354561   1111::20          1111::10          192.168.5.0/24      192.168.1.0/24      0x3c0ccb00    N     922 sec
 2     3758096386   1111::20          1111::10          fec0:5::/64         fec0:1::/64         0x23397400    N     880 sec
 2     2481958963   2424::101         5050::100                                                 0xe5eb0d00    N    1581 sec
IPsec-NoVTIPacket dropped due to IPsec input missing or admin-down VTI interface

VTI interface present on T0/T1 SR with mode as "vti" and we can find it by running

"get logical-router <T0/T1 SR UUID> interfaces"

edge1> get logical-router 220c92e1-343b-404f-b868-6c8365762d7b interfaces
Logical Router
UUID                                   VRF    LR-ID  Name                              Type                      
220c92e1-343b-404f-b868-6c8365762d7b   2      3      SR-ServerT0_AS                    SERVICE_ROUTER_TIER0      
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
    Interface     : cec70165-6e00-4ae8-93ef-b0338614a5e3
    Ifuid         : 295
    Mode          : vti
    Port-type     : vti
    IP/Mask       : 192.168.37.102/30;fec0:37::102/64(NA);fec0:37::50:56ff:fe01:400/64(NA);fe80::50:56ff:fe01:400/64(NA)
    Urpf-mode     : PORT_CHECK
    Admin         : up
    Op_state      : up



TX Interface Stats
 
Stats(Tx)DescriptionAction
BlockedPacket dropped due to interface being blocked/admin-down due to SR of the edge node is in standby stateSee Rx
FirewallPacket dropped due to firewallSee Rx
Frag-NeededPacket dropped due to the need of fragmentationDF bit is set but packet len is greater than than the MTU, check MTU configuration
No-neighborPacket dropped due to ARP failureThe neighbor IP has not reply the arp request, check the neighbor VM or router
No-MemoryPacket dropped due to shortage of memoryPotential mbuf/memory leaking
Edge1> get dataplane memory stats
Mon Feb 12 2024 UTC 03:39:39.950
Memory Usage
 
Available_entries             : 1024
Available_entries_in_cache    : 0
Cache_size_per_core           : 128
Name                          : jumbo_mbuf_pool
Size                          : 1024
No-Linked-PortPacket dropped due to failure in forwarding due to no linked peer lport

Check lrouter port configuration, it should have linked lswitch port like

nsx> get logical-router interface c32d0564-72f9-47c4-9d87-231eda4843ff       
Fri Jan 26 2024 PST 13:56:49.745
interface   : c32d0564-72f9-47c4-9d87-231eda4843ff
LS port     : 18ba9b76-23d9-4963-90e8-24b9d595a5fb
Rate-LimitPacket dropped due to egress traffic rate higher than configured QOS (traffic contract)

Check if egress traffic to the lrouter is much higher than the QOS config.
You can find the QOS configured using the below CLI,

Edge1> get logical-router 558ba208-8ae9-4b74-a379-55d94e3577ed qos-config
Mon Jan 29 2024 UTC 06:16:45.835
Logical Router
UUID           : 558ba208-8ae9-4b74-a379-55d94e3577ed
Direction      : Egress
Rate (Mbps)    : 1
Burst (bytes)  : 1
IPsecPacket dropped during IPsec output processing

Aggregated Tx drop counter for IPsec related drop, further check IPsec tunnel stats for particular drop reason using following command

"get ipsecvpn tunnel stats <IPsec tunnel/VTI UUID>" 

edge1> get ipsecvpn tunnel stats cec70165-6e00-4ae8-93ef-b0338614a5e3
Interface UID                      : 295
Interface UUID                     : cec70165-6e00-4ae8-93ef-b0338614a5e3
VTI UUID                           : cec70165-6e00-4ae8-93ef-b0338614a5e3
 
Stats
    Rx Pkts                            : 0             Tx Pkts                            : 0
    Rx Bytes                           : 0             Tx Bytes                           : 0
    Rx MSS Adjusted                    : 0             Tx MSS Adjusted                    : 0
    Rx MSS Ignored                     : 0             Tx MSS Ignored                     : 0
    Rx Drops                           : 0             Tx Drops                           : 0
    Rx Drop Crypto Failure             : 0             Tx Drop Crypto Failure             : 0
    Rx Drop Enqueue Failure            : 0             Tx Drop Enqueue Failure            : 0
    Rx Drop State Mismatch             : 0             Tx Drop State Mismatch             : 0
    Rx Drop Malformed                  : 0             Tx Drop Malformed                  : 0
    Rx Drop Proto Not Supported        : 0             Tx Drop Proto Not Supported        : 0
    Rx Drop Replay                     : 0             Tx Drop Seq Rollover               : 0
    Rx Drop Inner Malformed            : 0             Tx Drop Fragmentation Needed       : 0
    Rx Drop Policy Nomatch             : 0             Rekey Request Failure              : 0
    Rx Drop Auth Failure               : 0
    Rx Drop Zero Sequence Number       : 0
 
v6 Stats
    Rx Pkts                            : 3236          Tx Pkts                            : 3184
    Rx Bytes                           : 250276        Tx Bytes                           : 631712
    Rx MSS Adjusted                    : 0             Tx MSS Adjusted                    : 0
    Rx MSS Ignored                     : 0             Tx MSS Ignored                     : 0
    Rx Drops                           : 0             Tx Drops                           : 0
    Rx Drop Crypto Failure             : 0             Tx Drop Crypto Failure             : 0
    Rx Drop Enqueue Failure            : 0             Tx Drop Enqueue Failure            : 0
    Rx Drop State Mismatch             : 0             Tx Drop State Mismatch             : 0
    Rx Drop Malformed                  : 0             Tx Drop Malformed                  : 0
    Rx Drop Proto Not Supported        : 0             Tx Drop Proto Not Supported        : 0
    Rx Drop Replay                     : 0             Tx Drop Seq Rollover               : 0
    Rx Drop Inner Malformed            : 0             Tx Drop Fragmentation Needed       : 0
    Rx Drop Policy Nomatch             : 0             Rekey Request Failure              : 0
    Rx Drop Auth Failure               : 0
    Rx Drop Zero Sequence Number       : 0
----------------------------------------------------------------------------------------------------
IPsec-NoSAPacket dropped due to missing IPsec outbound or egress security association (SA)

Check Outbound and Inbound SAs installed by running below commands

"get ipsecvpn sad" or

"get ipsecvpn sad summary"

edge1> get ipsecvpn sad summary
Outbound SAs (6):
----------------------------------------------------------------------------------------------------------------------------
 VRF   Rule ID      Src IP            Dest IP           Src Subnet          Dest Subnet         SPI          NAT   Rem Life
----------------------------------------------------------------------------------------------------------------------------
 2     1408217139   5050::100         2424::101                                                 0xc6a6b127    N    1642 sec
 2     334475315    5050::100         2424::101                                                 0xcbe1de7e    N    1581 sec
 2     536870913    1111::10          1111::20          192.168.1.0/24      192.168.5.0/24      0x2e733800    N     922 sec
 5     536870927    192.168.51.100    20.20.20.101      192.168.2.0/24      192.168.7.0/24      0xce11f2c1    N    2279 sec
 2     1610612738   1111::10          1111::20          fec0:1::/64         fec0:5::/64         0x2786ae00    N     880 sec
 5     1610612752   192.168.51.100    20.20.20.101      fec0:2::/64         fec0:7::/64         0xc02800f4    N    2279 sec
 
Inbound SAs (6):
----------------------------------------------------------------------------------------------------------------------------
 VRF   Rule ID      Src IP            Dest IP           Src Subnet          Dest Subnet         SPI          NAT   Rem Life
----------------------------------------------------------------------------------------------------------------------------
 5     2684354575   20.20.20.101      192.168.51.100    192.168.7.0/24      192.168.2.0/24      0xbf806100    N    2279 sec
 2     3555700787   2424::101         5050::100                                                 0xc3061f00    N    1642 sec
 5     3758096400   20.20.20.101      192.168.51.100    fec0:7::/64         fec0:2::/64         0xef28dd00    N    2279 sec
 2     2684354561   1111::20          1111::10          192.168.5.0/24      192.168.1.0/24      0x3c0ccb00    N     922 sec
 2     3758096386   1111::20          1111::10          fec0:5::/64         fec0:1::/64         0x23397400    N     880 sec
 2     2481958963   2424::101         5050::100                                                 0xe5eb0d00    N    1581 sec
IPsec-NoVTIPacket dropped due to IPsec input missing or down VTI interface

VTI interface present on T0/T1 SR with mode as "vti" and we can find it by running

"get logical-router <T0/T1 SR UUID> interfaces"

edge1> get logical-router 220c92e1-343b-404f-b868-6c8365762d7b interfaces
Logical Router
UUID                                   VRF    LR-ID  Name                              Type                      
220c92e1-343b-404f-b868-6c8365762d7b   2      3      SR-ServerT0_AS                    SERVICE_ROUTER_TIER0      
Interfaces (IPv6 DAD Status A-DAD_Success, F-DAD_Duplicate, T-DAD_Tentative, U-DAD_Unavailable)
    Interface     : cec70165-6e00-4ae8-93ef-b0338614a5e3
    Ifuid         : 295
    Mode          : vti
    Port-type     : vti
    IP/Mask       : 192.168.37.102/30;fec0:37::102/64(NA);fec0:37::50:56ff:fe01:400/64(NA);fe80::50:56ff:fe01:400/64(NA)
    Urpf-mode     : PORT_CHECK
    Admin         : up
    Op_state      : up
IPsec-Policy-ErrorPacket dropped due to missing rule for IPsec processing. SPD look-up failed.

Check below command output there should not be any entry with tag "ipsec"

edge1> get firewall ded3f395-bd88-4c11-8472-75e6b1422837 ike policy
IKE policy count: 4
    Rule ID   : 536870913
    Policy    : out protocol any stateless from ip 192.168.1.0/24 to ip 192.168.5.0/24 secure keypolicy 00003400-2000-0000-2000-000100000000 tag 'ipsec'
 
    Rule ID   : 1610612738
    Policy    : out protocol any stateless from ip fec0:1::/64 to ip fec0:5::/64 secure keypolicy 00003400-2000-0000-2000-000200000000 tag 'ipsec'
IPsec-Policy-BlockPacket dropped due to IPsec rule with DROP as action

Check below command output for tag "DROP"

edge1> get firewall ded3f395-bd88-4c11-8472-75e6b1422837 ike policy
IKE policy count: 4
    Rule ID   : 536870913
    Policy    : out protocol any stateless from ip 192.168.1.0/24 to ip 192.168.5.0/24 secure keypolicy 00003400-2000-0000-2000-000100000000 tag 'ipsec'
 
    Rule ID   : 1610612738
    Policy    : out protocol any stateless from ip fec0:1::/64 to ip fec0:5::/64 secure keypolicy 00003400-2000-0000-2000-000200000000 tag 'ipsec'