This is a known issue impacting VMware NSX.
The following workaround procedures can be used to remove stale ports depending on the NSX release version.
The Python-based script (NSX-3.1.X-stale-lports-cleanup-v2.py) functions on NSX 4.x and may identify additional ports that can be cleaned up, although it may have slower performance than the Java-based cleanup script
NSX 3.2.1.x and higher
- Before proceeding, you must take an FTP based backup of the NSX Manager and ensure the passphrase is known.
- On one NSX Manager, copy the attached file, logical-migration.jar, to the location
/opt/vmware/upgrade-coordinator-tomcat/temp. - Run the script in dry run mode first, to see if it identifies stale logical ports (-DStaleLogicalPortCleanUp.dryRun=true):
Replace AdminPassword in the command with admin password for the system.
#java -Xms5g -Xmx10g \
-Dcorfu-property-file-path=/opt/vmware/upgrade-coordinator-tomcat/conf/ufo-factory.properties \
-Djava.io.tmpdir=/opt/vmware/upgrade-coordinator-tomcat/temp \
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector \
-Dlog4j.configurationFile=/opt/vmware/upgrade-coordinator-tomcat/conf/log4j2.xml \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Djava.util.logging.config.file=/opt/vmware/upgrade-coordinator-tomcat/conf/logging.properties \
-Dnsx-service-type=nsx-manager \
--add-opens=java.base/sun.security.provider=ALL-UNNAMED \
--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED \
--add-opens=java.base/java.util=ALL-UNNAMED \
--add-opens=java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED \
--add-opens=java.base/java.time=ALL-UNNAMED \
-DStaleLogicalPortCleanUp.dryRun=true \
-DStaleLogicalPortCleanUp.userName=admin \
-DStaleLogicalPortCleanUp.password='AdminPassword' \
-DStaleLogicalPortCleanUp.maxThreads=5 \
-DStaleLogicalPortCleanUp.batchSize=20 \
-DStaleLogicalPortCleanUp.maxTimeoutMinutes=30 \
-DStaleLogicalPortCleanUp.hostsSyncedBeforeMins=60 \
-DStaleLogicalPortCleanUp.portModificationThresholdTimeInMins=120 \
-cp /opt/vmware/upgrade-coordinator-tomcat/temp/logical-migration.jar \
com.vmware.nsx.management.migration.impl.StaleLogicalPortCleanUp
Note: On NSX 4.2.x, the first line of the output file has a warning "WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance."
This can be safely ignored and does not impact system functionality.
- After confirming the stale logical ports, run the script again without the dry run flag to remove the stale logical ports (-DStaleLogicalPortCleanUp.dryRun=false).
#java -Xms5g -Xmx10g \
-Dcorfu-property-file-path=/opt/vmware/upgrade-coordinator-tomcat/conf/ufo-factory.properties \
-Djava.io.tmpdir=/opt/vmware/upgrade-coordinator-tomcat/temp \
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector \
-Dlog4j.configurationFile=/opt/vmware/upgrade-coordinator-tomcat/conf/log4j2.xml \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
-Djava.util.logging.config.file=/opt/vmware/upgrade-coordinator-tomcat/conf/logging.properties \
-Dnsx-service-type=nsx-manager \
--add-opens=java.base/sun.security.provider=ALL-UNNAMED \
--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED \
--add-opens=java.base/java.util=ALL-UNNAMED \
--add-opens=java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED \
--add-opens=java.base/java.time=ALL-UNNAMED \
-DStaleLogicalPortCleanUp.dryRun=false \
-DStaleLogicalPortCleanUp.userName=admin \
-DStaleLogicalPortCleanUp.password='AdminPassword' \
-DStaleLogicalPortCleanUp.maxThreads=5 \
-DStaleLogicalPortCleanUp.batchSize=20 \
-DStaleLogicalPortCleanUp.maxTimeoutMinutes=30 \
-DStaleLogicalPortCleanUp.hostsSyncedBeforeMins=60 \
-DStaleLogicalPortCleanUp.portModificationThresholdTimeInMins=120 \
-cp /opt/vmware/upgrade-coordinator-tomcat/temp/logical-migration.jar \
com.vmware.nsx.management.migration.impl.StaleLogicalPortCleanUp
- Confirm stale logical ports have been removed.
NSX 3.1.x and higher
- Before proceeding, you must take an FTP based backup of the NSX Manager and ensure the passphrase is known.
- SSH as root user to one of the NSX manager's.
- Copy both attached files,
'NSX-3.1.X-stale-lports-cleanup-v2.py' and 'apiclient.py' to /root/ directory on any of the NSX Managers. - The script is run as follows
Scan all NSX ports:
python NSX-3.1.X-stale-lports-cleanup-v2.py
or
Scan a specific segment:
python NSX-3.1.X-stale-lports-cleanup-v2.py <segment name>
or
Scan a list of ports:
python NSX-3.1.X-stale-lports-cleanup-v2.py --input_list ports.txt
- The "
python"in front of the script name is designating python to be the interpreter and the command must be run with the above syntax - Regardless of the method used to launch the script, the user is prompted to select dry run mode or not. Dry run mode will detect stale ports but not delete them.
- The user will be prompted for the admin password
Is this a dry-run [y/n]: y
Dry run = True
Batch size = 100
Debug mode = False
Output file = nsx_port_cleanup_<date>.txt
Enter password for admin:
- If this is not a dry run, the user will be prompted to confirm deletion of detected stale ports
Number of stale logical-ports to delete: 1
1 port(s) are safe to delete. Delete them? [y/N]: y
logical port [<Port UUID>] is deleted
=================================================================
Summary:
Total logical switches scanned: 1
Total ports scanned: 7
Ports in DOWN state: 5
Ports safe to delete (orphaned): 1
Ports deleted: 1
=================================================================
Security Only Environments
- From NSX 4.1.1 the following API is available for security only deployments. If the stale ports exist in a security only environment, the following API can be used to sync with the Compute Manager:
POST /policy/api/v1/infra/sites/<site-id>/enforcement-points/<enforcement-point>/compute-managers/<compute-manager-id>/action/vds-full-sync
- To retrieve the Compute Manager ID run:
GET /api/v1/fabric/compute-managers
- For example, the following API can be used when using the default site, enforcement point and Compute Manager ID
12345678-####-####-####-123456789123:
POST /policy/api/v1/infra/sites/default/enforcement-points/default/compute-managers/12345678-####-####-####-123456789123/action/vds-full-sync
- Confirm if stale logical-ports are removed.
In NSX-T 3.1.x Security Only environment, use script_for_security_only_3-x.zip instead of the migration.jar and NSX-3.1.X-stale-lports-cleanup-v2.py tools.
- Before proceeding, you must take an FTP based backup of the NSX Manager and ensure the passphrase is known.
- Download script_for_security_only_3-x.zip and extract it.
- Copy both attached files, 'stale_logical_port_cleanup_for_security_only_setup.py' and 'apiclient.py' to /root/ directory on any of the NSX Managers.
- Run the script first in dry run mode first, to see if it identifies stale logical ports. Replace AdminPassword in the command with admin password for the system
#python3 stale_logical_port_cleanup_for_security_only_setup.py --user admin --password 'AdminPassword' --dry-run
- After confirming the stale logical ports, run the script again without the dry run flag to allow it to remove the stale logical ports
#python3 stale_logical_port_cleanup_for_security_only_setup.py --user admin --password 'AdminPassword'
- Confirm stale logical ports have been removed.