NSX-T error 5103 'logical router configuration realization error' and or 5105 'logical router FIB configuration realization error'

Products

• VMware vDefend Firewall

Issue/Introduction

• NSX-T version 3.2.x.
• North South service insertion is used.
• In NSX Manager UI you may see Tier-1 and Tier-0 Logical Router realization errors, Error Code = '5103', Error Message = 'logical router configuration realization error'.
• In NSX Manager UI you may also see Tier-1 and Tier-0 Logical Router realization errors, Error Code = '5105', Error Message = 'logical router FIB configuration realization error'.
• This can lead to North/South dataplane impact, affect new and existing Logical Routers.
• In the NSX-T manager /var/log/syslog you may see error similar to:

023-03-24T10:25:16.026Z manager1.corp.local NSX 6446 - [nsx@6876 comp="nsx-controller" level="WARNING" subcomp="domain"] No SrClusterId for Layer3SiPolicy message managed_resource { display_name: "396fa74d-####-####-####-b44732dea068"}service_insertion_policy { owner_id { c636581c-####-####-####-a5113b114c30#012 }}service_port_nhip { ipv4: 10.10.10.10}logical_router_id { 033cd93c-####-####-####-99d1980d48aa}

• In the NSX-T manager /var/log/cloudnet/nsx-ccp.log you may see error similar to: 

2023-03-27T15:47:20.962Z WARN Owl-worker-4 ExecutorTask 1542 - [nsx@6876 comp="nsx-controller" level="WARNING" subcomp="owl"] Failed to process dataUpdate for listener RedistributionAppImpl, error message: null, error stack:java.lang.NullPointerException

Environment

VMware NSX-T Data Center
VMware NSX

Cause

This is due to a Service Deployment having reference to a removed/deleted logical router.  
While attempting to publish a nestdb (internal DB) message, CCP (Central Control Plane) encountered a NPE (Null Pointer Exception) caused by the absence of the SrClusterId value. This resulted in the transaction being blocked.

Resolution

This issue is resolved in VMware NSX 3.2.4
This issue is resolved in VMware NSX 4.2.0

Workaround:

If you believe you have encountered this issue, please contact Broadcom Support and refer to this KB article.

Additional Information

Impact/Risks:

Logical routers may go in to an unknown state with realization errors.
North/South traffic may cease and East/West traffic may also be affected.