Publishing Distributed Firewall (DFW) rules fails after referenced object is deleted in VMware NSX for vSphere 6.1.x and 6.2.x
book
Article ID: 339084
calendar_today
Updated On:
Products
VMware vDefend Firewall
Issue/Introduction
Symptoms:
Publishing new Distributed Firewall rules fails
You see the error:
Configuration contain invalid rules. A rule becomes invalid if grouping objects, service or service group used in the rule are deleted or service profile is not assigned to a partner security services rule. This configuration cannot be published.
Force Sync fails
In the /home/secureall/secureall/logs/vsm.log file on the NSX Manager, you see entries similar to:
2015-05-11 09:19:29.675 EST ERROR TaskFrameworkExecutor-11 FirewallConfigurationSyncService:290 - Update operation for SG apply has failed. Marking Service Composer Firewall config as out of sync. com.vmware.vshield.vsm.exceptions.ObjectNotFoundException: core-services:202:The requested object : 501d40e1-xxxx-xxxx-xxxxxxxx.000 could not be found. Object identifiers are case sensitive.
In the Firewall DCN Handler code, you see entries similar to:
2015-05-11 09:19:35.188 EST ERROR DCNPool-6 BatchingBatcher:73 - Exception executing batch: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [3]; actual row count: 0; expected: 1 at com.vmware.vshield.firewall.service.impl.FirewallServiceImpl.setAppliedToInvalid(FirewallServiceImpl.java:2124)
In the RelationshipNotificationHandler code, you see entries similar to:
2015-05-06 21:38:54.141 EST ERROR DCNPool-3 RelationshipManagerNotificationHandler:219 - Could not mark the vm cache as stale for securitygroup-56
org.springframework.orm.jpa.JpaOptimisticLockingFailureException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is javax.persistence.OptimisticLockException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:310) at java.lang.Thread.run(Unknown Source)
2015-05-06 23:54:17.991 EST ERROR DCNPool-4 RelationshipManagerNotificationHandler:219 - Could not mark the vm cache as stale for securitygroup-46
org.springframework.orm.jpa.JpaOptimisticLockingFailureException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is javax.persistence.OptimisticLockException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1 at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:310)
In the DynamicCriteriaHandler code, you see entries similar to:
2015-05-06 23:55:00.050 EST ERROR DCNPool-4 AbstractFlushingEventListener:324 - Could not synchronize database state with session org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.vmware.vshield.vsm.dynamicmembership.model.DynamicCriteria#dynamiccriteria-180]
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware NSX for vSphere 6.1.x VMware NSX for vSphere 6.2.x
Cause
This issue occurs when the OptimisticLocking exception is thrown in the DCN Handler of the Firewall, DynamicCriteriaHandler and RelationshipNotificationHandler.
Resolution
This issue is resolved in VMware NSX for vSphere 6.2.4
To work around this issue when you are unable to upgrade:
Note: Take a snapshot of the NSX Manager appliance before proceeding. Ensure the snapshot is committed once you are satisfied with the changes.
Get the complete Firewall Configuration using REST API. https://NSX Manager ip/api/4.0/firewall/globalroot-0/config
Note: Document the Section Ids that has the invalid entries. (Service Composer plus default section Ids).
Get Firewall Configuration for the section ids that has invalid entries. For example, If section Id for default is 1003:
Remove invalid Vnic from appliedTo. Get Firewall Configuration for service Id belongs to Service composer,and remove invalid entry when reposting the configuration for respective section(PUT call from above Step). If vNic is invalid in AppliedTo column, remove <appliedto> for invalid vnic during reposting the configuration