NSX Firewall rules are deleted
search cancel

NSX Firewall rules are deleted

book

Article ID: 399804

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • The rules are authored by TKG.
  • The rules were present allowing the system to pass traffic and at some point they are found to be "missing".
  • Log entries will show that there was a deletion request and the user is wcp-cluster-user-<UUID>

Environment

VMware NSX

Cause


Very simple high level diagram of the relationship of NSX, NCP, and TKG.

  • The TKG masters will send API call destined for the NSX manager to execute.  These calls are sent to the NSX Container Plugin (NCP).
  • The NCP does not have the ability to initiate any API calls on its own.
  • NCP receives calls from, in this case the TKG cluster, to execute an API call on behalf of a TKG Pod.
  • NCP then proxies the API to NSX manager.
  • There are no log entries within the NCP that will  record what source requested the API call to be executed by NSX manager.
  • For the scope of this article the firewall rules are authored by the TKG components. No manually created rules exist.

    NSX log entries that will show what was executed have the following identifiers:

2025-05-30T11:54:30.308Z  INFO http-nio-127.0.0.1-7440-exec-11 PreAuthenticationFilter 5436 PreAuthenticationFilter setting username to wcp-cluster-user-<UUID> from x-nsx-username header.

2025-05-30T11:54:30.447Z  INFO providerTaskExecutor-1-53 AbstractRuleSectionProvider 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Processing cMap /infra/domains/domain-c9:<UUID>/security-policies/ds_domain-c9:<UUID> with 0 cEntries, rzPath /infra/realized-state/enforcement-points/default/firewalls/firewall-sections/domain-c9:<UUID>.ds_domain-c9:<UUID>, state UNREALIZED, markedForDelete [true], existing FirewallSection/03c65b6f-####-####-####-16360579bec4

2025-05-30T11:54:30.448Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000000c33 in Service Layer

2025-05-30T11:54:30.448Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001401 in Service Layer

2025-05-30T11:54:30.448Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001bfa in Service Layer

2025-05-30T11:54:30.448Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001fe9 in Service Layer

2025-05-30T11:54:30.448Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001fed in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001ff6 in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001004 in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001005 in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001006 in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001003 in Service Layer

2025-05-30T11:54:30.449Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted DS Rule 00000000-0000-0000-0000-000000001007 in Service Layer

2025-05-30T11:54:30.462Z  INFO http-nio-127.0.0.1-7440-exec-24 PreAuthenticationFilter 5436 PreAuthenticationFilter setting username to wcp-cluster-user-<UUID> from x-nsx-username header.

2025-05-30T11:54:30.468Z  INFO providerTaskExecutor-1-53 FirewallServiceImpl 5436 FIREWALL [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Firewall section priority operation enqueued AbstractSectionPriorityTask [tN=FirewallSectionPriorityTask, id=731b7c34-####-####-####-3b3ebed82ab6, cId=e6b33ca8-####-####-####-0242ac130003, cTime=1748606070449, opList (1) [opType=SPUT_DELETE, anchorId=, currSecId=03c65b6f-####-####-####-16360579bec4, ], ]

2025-05-30T11:54:30.468Z  INFO providerTaskExecutor-1-53 AbstractDao 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Delete relationships for AbstractSection FirewallSection/03c65b6f-####-####-####-16360579bec4

2025-05-30T11:54:30.468Z  INFO providerTaskExecutor-1-53 AbstractDao 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Delete Abstract Section for Id : FirewallSection/03c65b6f-####-####-####-16360579bec4

2025-05-30T11:54:30.468Z  INFO providerTaskExecutor-1-53 AbstractService 5436 SERVICE [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted Section FirewallSection/03c65b6f-####-####-####-16360579bec4 along with 11 rule(s) in Service Layer

2025-05-30T11:54:30.471Z  INFO providerTaskExecutor-1-53 PolicyServiceImpl 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Permanently deleted entity /infra/realized-state/enforcement-points/default/firewalls/firewall-sections/domain-c9:<UUID>.ds_domain-c9:<UUID>

2025-05-30T11:54:30.507Z  INFO providerTaskExecutor-1-53 PolicyServiceImpl 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Permanently deleted entity /infra/domains/domain-c9:<UUID>/security-policies/ds_domain-c9:<UUID>

2025-05-30T11:54:30.512Z  INFO providerTaskExecutor-1-53 PolicyRealizedStateServiceImpl 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] deleted object /infra/domains/domain-c9:<UUID>/security-policies/ds_domain-c9:<UUID>

2025-05-30T11:54:30.513Z  INFO providerTaskExecutor-1-53 PolicyRealizedStateServiceImpl 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted publish status for GPRR /infra/realized-state/enforcement-points/default/firewalls/firewall-sections/domain-c9:<UUID>.ds_domain-c9:<UUID>

2025-05-30T11:54:30.513Z  INFO providerTaskExecutor-1-53 AbstractRuleSectionProvider 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted section FirewallSection/03c65b6f-####-####-####-16360579bec4 with rzPath /infra/realized-state/enforcement-points/default/firewalls/firewall-sections/domain-c9:<UUID>.ds_domain-c9:<UUID> for cMap /infra/domains/domain-c9:<UUID>/security-policies/ds_domain-c9:<UUID> in FirewallRuleSectionProvider

2025-05-30T11:54:30.521Z  INFO http-nio-127.0.0.1-7440-exec-23 PreAuthenticationFilter 5436 PreAuthenticationFilter setting username to wcp-cluster-user-<UUID> from x-nsx-username header.

2025-05-30T11:54:30.533Z  INFO http-nio-127.0.0.1-7440-exec-23 AutoDraftCreationAspect 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="b15792a3-####-####-####-307ad71e5ba9" subcomp="manager" username="wcp-cluster-user-<UUID>"] Performing auto draft creation: void com.vmware.nsx.management.policy.facade.DfwSecurityPolicyFacadeImpl.deleteSecurityPolicyForDomain(String, String). Request id: b15792a3-####-####-####-307ad71e5ba9

2025-05-30T11:54:30.549Z  INFO http-nio-127.0.0.1-7440-exec-23 PolicyServiceImpl 5436 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="b15792a3-####-####-####-307ad71e5ba9" subcomp="manager" username="wcp-cluster-user-<UUID>"] Created new map of InfraTreeBuilders for request b15792a3-####-####-####-307ad71e5ba9

 

Resolution

Manually recreate the firewall rules.
Add a Firewall Rule in Manager Mode

Powered by Wolken Software