NSX DFW rules are not pushed to ESXi hosts in federated environment when Local Manager is upgraded to a later version than Global Manager
Article ID: 319111
Updated On:
Products
VMware NSX
Issue/Introduction
Symptoms:
- Local Managers have been upgraded to a later version than the Global Manager, which is the normal upgrade order when going from 3.1.x to 3.2.x. This issue is observed when Local Managers are upgraded from 3.1.x to 3.2.x releases.
- Service Entries created by users on the Global Manager fail to realize on the Local Manager.
- DFW rules with unrealized Service Entries are not published to ESXi hosts.
- NullPointerException (java.lang.NullPointerException) is observed when the provider attempts realization of the SERVICE_ENTRY.
/var/log/syslog on the Local Manager shows a NullPointerException with the errorId PROVIDER_INVOCATION_FAILURE:
20xx-xx-xxTxx:xx:xx.xxxZ LM-2 NSX 4640 POLICY [nsx@6876 comp="nsx-manager" errorCode="PM0" level="ERROR" subcomp="manager"] Created alarm Alarm [policyPath=/global-infra/realized-state/enforcement-points/default/services/nsservices/SERVICE:Service-User-Created-2,SERVICE_ENTRY:Service-Entry-User-Created/alarms/<UUID>, message=java.lang.NullPointerException,errorId=PROVIDER_INVOCATION_FAILURE, path=null, apiError=null, sourceSiteId=null].
- Local Managers have been upgraded to a later version than the Global Manager, which is the normal upgrade order when going from 3.1.x to 3.2.x. This issue is observed when Local Managers are upgraded from 3.1.x to 3.2.x releases.
- Service Entries created by users on the Global Manager fail to realize on the Local Manager.
- DFW rules with unrealized Service Entries are not published to ESXi hosts.
- NullPointerException (java.lang.NullPointerException) is observed when the provider attempts realization of the SERVICE_ENTRY.
/var/log/syslog on the Local Manager shows a NullPointerException with the errorId PROVIDER_INVOCATION_FAILURE:
20xx-xx-xxTxx:xx:xx.xxxZ LM-2 NSX 4640 POLICY [nsx@6876 comp="nsx-manager" errorCode="PM0" level="ERROR" subcomp="manager"] Created alarm Alarm [policyPath=/global-infra/realized-state/enforcement-points/default/services/nsservices/SERVICE:Service-User-Created-2,SERVICE_ENTRY:Service-Entry-User-Created/alarms/<UUID>, message=java.lang.NullPointerException,errorId=PROVIDER_INVOCATION_FAILURE, path=null, apiError=null, sourceSiteId=null].
Environment
VMware NSX-T Data Center 3.x
VMware NSX-T Data Center
VMware NSX-T Data Center
Cause
When a new Service Entry is created on the Global Manager on 3.1.x, the is_default flag is set to null on both the Global Manager and Local Manager.
On Local Managers upgraded to 3.2.x, new Service Entry is_default flags are set to False.
In this scenario, after the Local Manager is upgraded to 3.2.x, the Service Entry is_default flag is accessed and a Null Pointer Exception is thrown. This prevents Service Entries from being realized. All new Service Entries created from the Global Manager will not be realized because the is_default is always set to null on the Global Manager on 3.1.x. Service Entries created from Local Manager will not be impacted, because the is_default flag is correctly set to false when they are created on the local Manager.
On Local Managers upgraded to 3.2.x, new Service Entry is_default flags are set to False.
In this scenario, after the Local Manager is upgraded to 3.2.x, the Service Entry is_default flag is accessed and a Null Pointer Exception is thrown. This prevents Service Entries from being realized. All new Service Entries created from the Global Manager will not be realized because the is_default is always set to null on the Global Manager on 3.1.x. Service Entries created from Local Manager will not be impacted, because the is_default flag is correctly set to false when they are created on the local Manager.
Resolution
This issue is resolved in NSX-T version 3.2.3 and later releases.
Workaround:
Workaround steps:
1. Upgrade the Global Manager to match the Local Manager version.
2. Edit the Service Entries which failed realization from the Global Manager and republish them with or without any changes.
Open a Service Request with VMware support if there are a huge number of Service Entries in an unrealized state.
Workaround:
Workaround steps:
1. Upgrade the Global Manager to match the Local Manager version.
2. Edit the Service Entries which failed realization from the Global Manager and republish them with or without any changes.
Open a Service Request with VMware support if there are a huge number of Service Entries in an unrealized state.
Additional Information
Impact/Risks:
DFW rules and Services are not realized on the upgraded Local Manager.
DFW rules and Services are not realized on the upgraded Local Manager.
Feedback
Yes
No
Powered by
