Publishing Distributed Firewall (DFW) rules fails after referenced object is deleted in VMware NSX for vSphere 6.1.x and 6.2.x
search cancel

Publishing Distributed Firewall (DFW) rules fails after referenced object is deleted in VMware NSX for vSphere 6.1.x and 6.2.x

book

Article ID: 339084

calendar_today

Updated On:

Products

VMware vDefend Firewall

Issue/Introduction

Symptoms:

  • Publishing new Distributed Firewall rules fails
  • You see the error:

    Configuration contain invalid rules. A rule becomes invalid if grouping objects, service or service group used in the rule are deleted or service profile is not assigned to a partner security services rule. This configuration cannot be published.
  • Force Sync fails
  • In the /home/secureall/secureall/logs/vsm.log file on the NSX Manager, you see entries similar to:

    2015-05-11 09:19:29.675 EST ERROR TaskFrameworkExecutor-11 FirewallConfigurationSyncService:290 - Update operation for SG apply has failed. Marking Service Composer Firewall config as out of sync.
    com.vmware.vshield.vsm.exceptions.ObjectNotFoundException:
    core-services:202:The requested object : 501d40e1-xxxx-xxxx-xxxxxxxx.000 could not be found. Object identifiers are case sensitive.
  • In the Firewall DCN Handler code, you see entries similar to:

    2015-05-11 09:19:35.188 EST ERROR DCNPool-6 BatchingBatcher:73 - Exception executing batch:
    org.hibernate.StaleStateException: Batch update returned unexpected row count from update [3]; actual row count: 0; expected: 1
    at com.vmware.vshield.firewall.service.impl.FirewallServiceImpl.setAppliedToInvalid(FirewallServiceImpl.java:2124)
  • In the RelationshipNotificationHandler code, you see entries similar to:
    • 2015-05-06 21:38:54.141 EST ERROR DCNPool-3 RelationshipManagerNotificationHandler:219 - Could not mark the vm cache as stale for securitygroup-56
    org.springframework.orm.jpa.JpaOptimisticLockingFailureException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is javax.persistence.OptimisticLockException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
    at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:310)
    at java.lang.Thread.run(Unknown Source)

    • 2015-05-06 23:54:17.991 EST ERROR DCNPool-4 RelationshipManagerNotificationHandler:219 - Could not mark the vm cache as stale for securitygroup-46
    org.springframework.orm.jpa.JpaOptimisticLockingFailureException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is javax.persistence.OptimisticLockException: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
    at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:310)
  • In the DynamicCriteriaHandler code, you see entries similar to:

    2015-05-06 23:55:00.050 EST ERROR DCNPool-4 AbstractFlushingEventListener:324 - Could not synchronize database state with session
    org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.vmware.vshield.vsm.dynamicmembership.model.DynamicCriteria#dynamiccriteria-180]

    Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.



Environment

VMware NSX for vSphere 6.1.x
VMware NSX for vSphere 6.2.x

Cause

This issue occurs when the OptimisticLocking exception is thrown in the DCN Handler of the Firewall, DynamicCriteriaHandler and RelationshipNotificationHandler.

Resolution

This issue is resolved in VMware NSX for vSphere 6.2.4

To work around this issue when you are unable to upgrade:

Note:
Take a snapshot of the NSX Manager appliance before proceeding. Ensure the snapshot is committed once you are satisfied with the changes.
  1. Get the complete Firewall Configuration using REST API.
    https://NSX Manager ip/api/4.0/firewall/globalroot-0/config

    Note: Document the Section Ids that has the invalid entries. (Service Composer plus default section Ids).

  2. Get Firewall Configuration for the section ids that has invalid entries. For example, If section Id for default is 1003:

    https://NSX Manager ip//api/4.0//firewall/globalroot-0/config/layer3sections/1003
  3. Correct the invalid entry and post the corrected section as:

    PUT https://<nsx-manager>/api/4.0//firewall/globalroot-0/config/layer3sections/1003

    Note: If in the default section source the virtual machine is invalid, remove <sources> tag during configuration reposting.
    For example:

    <sources excluded="false">
    <source>
    <name>xxxxxxxx</name>
    <value>vm-17</value>
    <type>VirtualMachine</type>
    <isValid>true</isValid>


  4. Remove invalid Vnic from appliedTo. Get Firewall Configuration for service Id belongs to Service composer,and remove invalid entry when reposting the configuration for respective section(PUT call from above Step).
    If vNic is invalid in AppliedTo column, remove <appliedto> for invalid vnic during reposting the configuration

    For example:

    <appliedTo>
    <name>xxxxxxxx - Network adapter 1</name>
    <value>5039218c-xxxx-xxxx-xxxxxxxx.000</value>
    <type>Vnic</type>
    <isValid>true</isValid>
    </appliedTo</font>>
  5. Go to Service Composer page and click Synchronize Firewall Config. This makes the Service Composer in Sync.



Additional Information



Powered by Wolken Software