Replication errors during NSX upgrade in Cross vCenter environment
Article ID: 325714
Updated On:
Products
VMware NSX
Issue/Introduction
This article is to highlight the documentation in regards to the sync issues displayed.
Symptoms:
After upgrading the Primary NSX Manager, and before upgrading the Secondary NSX Manager(s):
To further identify this condition, the following matches can be found in the NSX Manager logs:
<appguid><appGuidId>1697</appGuidId><appGuidName>GITHUB</appGuidName></appguid> on VSM 10.140.0.220
org.springframework.web.client.HttpClientErrorException: 404 Not Found
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:85) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:708) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:661) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:621) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:388) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at com.vmware.vshield.replicator.utils.VsmRestClient.post(VsmRestClient.java:148) ~[classes/:?]
at com.vmware.vshield.replicator.utils.ReplicatorUtils.executeRestApi(ReplicatorUtils.java:53) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.executeRestApi(DefaultReplicationProvider.java:302) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleCreate(DefaultReplicationProvider.java:74) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleCreateInternal(DefaultReplicationProvider.java:348) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleFullSync(DefaultReplicationProvider.java:187) [classes/:?]
at com.vmware.vshield.replicator.queuemgmt.SecondaryReplicationQueue$SecondaryEventDispatcher.run(SecondaryReplicationQueue.java:156) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Note: This log excerpt is an example. Date, time, and environmental variables may vary depending on your environment.
Symptoms:
After upgrading the Primary NSX Manager, and before upgrading the Secondary NSX Manager(s):
- You see the following sync issues in vSphere Web Client under Networking & Security / Installation and Upgrade:
To further identify this condition, the following matches can be found in the NSX Manager logs:
- In localhost_acess_log_date.txt:
- In replicator.log:
<appguid><appGuidId>1697</appGuidId><appGuidName>GITHUB</appGuidName></appguid> on VSM 10.140.0.220
org.springframework.web.client.HttpClientErrorException: 404 Not Found
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:85) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:708) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:661) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:621) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:388) ~[spring-web-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at com.vmware.vshield.replicator.utils.VsmRestClient.post(VsmRestClient.java:148) ~[classes/:?]
at com.vmware.vshield.replicator.utils.ReplicatorUtils.executeRestApi(ReplicatorUtils.java:53) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.executeRestApi(DefaultReplicationProvider.java:302) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleCreate(DefaultReplicationProvider.java:74) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleCreateInternal(DefaultReplicationProvider.java:348) [classes/:?]
at com.vmware.vshield.replicator.providers.DefaultReplicationProvider.handleFullSync(DefaultReplicationProvider.java:187) [classes/:?]
at com.vmware.vshield.replicator.queuemgmt.SecondaryReplicationQueue$SecondaryEventDispatcher.run(SecondaryReplicationQueue.java:156) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Note: This log excerpt is an example. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware NSX for vSphere 6.4.x
Cause
The sync issues occur because the REST API call used for the specific object exists in the upgraded Primary NSX Manager but not in the non-upgraded Secondary NSX Manager(s).
Resolution
As documented in NSX 6.4 Upgrade Guide:
Proceed with the upgrade of the Secondary NSX Manager(s). The sync issues will be automaticall cleared after all Secondary NSX Managers are upgraded.
If you are logged in to the vSphere Web Client during the upgrade, you will see synchronization issue warnings on the Networking and Security > Installation and Upgrade > Management page. This is because you have NSX Manager appliances with different versions of NSX. You must upgrade the secondary NSX Manager appliances before proceeding with any other part of the upgrade.
Proceed with the upgrade of the Secondary NSX Manager(s). The sync issues will be automaticall cleared after all Secondary NSX Managers are upgraded.
Additional Information
Impact/Risks:
This does not cause any impact.
This does not cause any impact.
Feedback
Yes
No
Powered by
