NSX 4.1.2 upgrade prechecks fail with "NSX Manager upgrade dry run failed"
Article ID: 324241
Updated On:
Products
VMware NSX Networking
Issue/Introduction
Symptoms:
2023-10-02T14:52:58.244Z ERROR ViewsGarbageCollector AbstractView 3162186 getLayoutUninterruptibly: Encountered error. Aborting layoutHelper
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
2023-10-02T14:56:05.371Z ERROR main Migration 3162186 - [nsx@6876 comp="nsx-manager" errorCode="MP217" level="ERROR" subcomp="manager"] Migration failed
java.lang.RuntimeException: java.util.concurrent.TimeoutException
- NSX Upgrade to 4.1.2 from version 3.2.x, 4.0.x or 4.1.x.
- NSX Upgrade Pre-Checks fail with "NSX Manager upgrade dry run failed. Do not proceed with the upgrade."
- NSX Manager logs may show an error similar to this example
/var/log/syslog
2023-09-21T05:14:46.957Z ERROR providerTaskExecutor-87 PolicyProviderUtil 4917 POLICY [nsx@6876 comp="nsx-manager" errorCode="PM0" level="ERROR" subcomp="manager"] Created alarm Alarm [policyPath=/infra/realized-state/enforcement-points/default/logical-ports/infra-ad210bdc-7ed2-4c63-96f0-09a0d5e63a90-default:af6a1f82-8a20-44f2-a199-4aa6ccc812cd-lp/alarms/9ca05cac-0dcd-4575-85c5-f71688edfd61, message=TX ABORT | Snapshot Time = Token(epoch=2164, sequence=3391905417) | Failed Transaction ID = e44840b4-684b-8e77-c4e5-3c63c9ba8011 | Offending Address = 3391905445 | Conflict Key = 3E0923C968CFCDDC | Conflict Stream = dabf8af4-9eb6-3374-9a18-d273ed7132e9 | Cause = CONFLICT | Time = 194 ms,errorId=PROVIDER_INVOCATION_FAILURE, path=null, apiError=null, sourceSiteId=null].
2023-09-21T05:14:46.957Z ERROR providerTaskExecutor-87 PolicyProviderUtil 4917 POLICY [nsx@6876 comp="nsx-manager" errorCode="PM0" level="ERROR" subcomp="manager"] Created alarm Alarm [policyPath=/infra/realized-state/enforcement-points/default/logical-ports/infra-ad210bdc-7ed2-4c63-96f0-09a0d5e63a90-default:af6a1f82-8a20-44f2-a199-4aa6ccc812cd-lp/alarms/9ca05cac-0dcd-4575-85c5-f71688edfd61, message=TX ABORT | Snapshot Time = Token(epoch=2164, sequence=3391905417) | Failed Transaction ID = e44840b4-684b-8e77-c4e5-3c63c9ba8011 | Offending Address = 3391905445 | Conflict Key = 3E0923C968CFCDDC | Conflict Stream = dabf8af4-9eb6-3374-9a18-d273ed7132e9 | Cause = CONFLICT | Time = 194 ms,errorId=PROVIDER_INVOCATION_FAILURE, path=null, apiError=null, sourceSiteId=null].
/var/log/upgrade-coordinator/logical-migration.log
2023-10-02T14:52:58.244Z ERROR ViewsGarbageCollector AbstractView 3162186 getLayoutUninterruptibly: Encountered error. Aborting layoutHelper
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
2023-10-02T14:56:05.371Z ERROR main Migration 3162186 - [nsx@6876 comp="nsx-manager" errorCode="MP217" level="ERROR" subcomp="manager"] Migration failed
java.lang.RuntimeException: java.util.concurrent.TimeoutException
/image contains a dump file migration_oom.hprof.gz
Environment
VMware NSX-T Data Center
VMware NSX-T Data Center 4.x
VMware NSX-T Data Center 4.x
Cause
This issue occurs when the NSX Manager database contains a large number of stale port entries which causes unexpectedly high transaction times.
Note upgrades from greenfield deployments of 4.1.1 are not impacted by this issue.
Note upgrades from greenfield deployments of 4.1.1 are not impacted by this issue.
Resolution
This is a known issue impacting NSX, currently there is no resolution.
Workaround:
Please open a Support Request with VMware to get guidance on a workaround for this issue.
Workaround:
Please open a Support Request with VMware to get guidance on a workaround for this issue.
Feedback
Yes
No
Powered by
