Scripted cleanup of stale logical ports on NSX segments

Products

• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

The purpose of this KB is to provide a script that can be used to clean up stale ports in NSX environments.

Environment

VMware NSX

Cause

Stale ports may arise in an environment for different reasons, automatic handling is planned for a future version.

Resolution

This is a known issue impacting VMware NSX.

The following workaround procedures can be used to remove stale ports depending on the NSX release version.

NSX 3.2.1.x and higher

1. Before proceeding, you must take an FTP based backup of the NSX Manager and ensure the passphrase is known.
2. On one NSX Manager, copy the attached file, logical-migration.jar, to the location /opt/vmware/upgrade-coordinator-tomcat/temp
3. Run the script first in dry run mode first, to see if it identifies stale logical ports (-DStaleLogicalPortCleanUp.dryRun=true)
   Replace AdminPassword in the command with admin password for the system
   
   #java -Xms5g -Xmx10g --add-opens=java.base/java.util=ALL-UNNAMED -Dcorfu-property-file-path=/opt/vmware/upgrade-coordinator-tomcat/conf/ufo-factory.properties -Djava.io.tmpdir=/opt/vmware/upgrade-coordinator-tomcat/temp -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dlog4j.configurationFile=/opt/vmware/upgrade-coordinator-tomcat/conf/log4j2.xml -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/opt/vmware/upgrade-coordinator-tomcat/conf/logging.properties -Dnsx-service-type=nsx-manager -DStaleLogicalPortCleanUp.dryRun=true -DStaleLogicalPortCleanUp.userName=admin -DStaleLogicalPortCleanUp.password='AdminPassword'  -DStaleLogicalPortCleanUp.maxThreads=5  -DStaleLogicalPortCleanUp.batchSize=20  -DStaleLogicalPortCleanUp.maxTimeoutMinutes=30 -cp /opt/vmware/upgrade-coordinator-tomcat/temp/logical-migration.jar com.vmware.nsx.management.migration.impl.StaleLogicalPortCleanUp

Note on NSX 4.2.x, the first line of the output file has a warning "WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance."
   This can be safely ignored and does not impact the functionality.
   
   
4. After confirming the stale logical ports, run the script again without the dry run flag to allow it to remove the stale logical ports (where -DStaleLogicalPortCleanUp.dryRun=false).
   
   #java -Xms5g -Xmx10g --add-opens=java.base/java.util=ALL-UNNAMED -Dcorfu-property-file-path=/opt/vmware/upgrade-coordinator-tomcat/conf/ufo-factory.properties -Djava.io.tmpdir=/opt/vmware/upgrade-coordinator-tomcat/temp -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dlog4j.configurationFile=/opt/vmware/upgrade-coordinator-tomcat/conf/log4j2.xml -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/opt/vmware/upgrade-coordinator-tomcat/conf/logging.properties -Dnsx-service-type=nsx-manager -DStaleLogicalPortCleanUp.dryRun=false -DStaleLogicalPortCleanUp.userName=admin -DStaleLogicalPortCleanUp.password='AdminPassword'  -DStaleLogicalPortCleanUp.maxThreads=5  -DStaleLogicalPortCleanUp.batchSize=20  -DStaleLogicalPortCleanUp.maxTimeoutMinutes=30 -cp /opt/vmware/upgrade-coordinator-tomcat/temp/logical-migration.jar com.vmware.nsx.management.migration.impl.StaleLogicalPortCleanUp
   
   
5. Confirm stale logical ports have been removed.

NSX 3.1.x

1. Before proceeding, you must take an FTP based backup of the NSX Manager and ensure the passphrase is known.
2. SSH as root user to one of the NSX manager's.
3. Copy both attached files, 'NSX-3.1.X-stale-lports-cleanup-v2.py' and 'apiclient.py' to /root/ directory on any of the NSX Managers.
4. The script is run as follows
   
   Scan all NSX ports:
   python NSX-3.1.X-stale-lports-cleanup-v2.py

or

   Scan a specific segment:
   python NSX-3.1.X-stale-lports-cleanup-v2.py <segment UUID>
Note: To find a segment UUID, on the NSX manager as admin user, run: get logical-switches

or

   Scan a list of ports:
   python NSX-3.1.X-stale-lports-cleanup-v2.py --input_list ports.txt


   • The "python"in front of the script name is designating python to be the interpreter and the command must be run with the above syntax
   • Regardless of the method used to launch the script, the user is prompted to select dry run mode or not. Dry run mode will detect stale ports but not delete them.
   • The user will be prompted for the admin password
     
     Is this a dry-run [y/n]: y
Dry run = True
Batch size = 100
Debug mode = False
Output file = nsx_port_cleanup_<date>.txt
Enter password for admin:
     
     
5. If this is not a dry run, the user will be prompted to confirm deletion of detected stale ports
   
   

   Number of stale logical-ports to delete: 1

   1 port(s) are safe to delete. Delete them? [y/N]: y
logical port [<Port UUID>] is deleted

=================================================================
Summary:
Total logical switches scanned: 1
Total ports scanned: 7
Ports in DOWN state: 5
Ports safe to delete (orphaned): 1
Ports deleted: 1
=================================================================

Security Only Environments

1. From NSX 4.1.1 the following API is available for security only deployments. If the stale ports exist in a security only environment, the following API can be used to sync with the Compute Manager:
   
   POST /policy/api/v1/infra/sites/<site-id>/enforcement-points/<enforcement-point>/compute-managers/<compute-manager-id>/action/vds-full-sync
   
   
2. To retrieve the Compute Manager ID run:
   
   GET /api/v1/fabric/compute-managers
   
   
3. For example, the following API can be used when using the default site, enforcement point and Compute Manager ID 12345678-####-####-####-123456789123:
   
   POST /policy/api/v1/infra/sites/default/enforcement-points/default/compute-managers/12345678-####-####-####-123456789123/action/vds-full-sync
   
   
4. Confirm if stale logical-ports are removed.

Additional Information

Note: Running this script on a Global Manager, in a Federated environment will return results saying that no stale logical ports are found. The stale ports created for the Global segments will be automatically cleaned after cleaning the stale ports from the associated Local Manager.


Related Knowledge Base articles

Manual port cleanup of stale ports can be found in KB - Steps to remove stale logical-port(s) in NSX-T after attempted delete