While preforming a V2T migration the migration-coordinator hangs or spins indefinitely while attempting to load the Resolve Configuration (L3-L7 services) page.
VMware NSX
The size of the feedback_request_l3_l7.json file is huge, (93MB) in a particular customer scenario, due to the fact that the details field in the "GROUPINGOBJECTS" vertical has too many values. The app is crashing when trying to load this data to memory to serve the REST API, which results in 500 internal server errors.
Relevant log locations and log entries are as follows:
Log Locations:
/var/log/migration-coordinator/v2t/cm.log
/var/log/migration-coordinator/migration-coordinator.log
/var/log/migration-coordinator/localhost_access_log.txt
/var/log/migration-coordinator/v2t/collection.log
Log Entries:
2025-04-16T15:47:48.148Z INFO http-nio-127.0.0.1-7450-exec-2 FacadeInterceptorHelperImpl 589938 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Caught error in facade interceptorjava.lang.NullPointerException: null at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackRequestFile(V2tFeedbackServiceImpl.java:458) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackRequests(V2tFeedbackServiceImpl.java:102) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackSummary(V2tFeedbackServiceImpl.java:200) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl.getFeedbackSummary_aroundBody24(MigrationFacadeImpl.java:375) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl$AjcClosure25.run(MigrationFacadeImpl.java:1) ~[libmc-core.jar:?] at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.9.0.jar:1.9.0] at com.vmware.nsx.management.upgrade.facade.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:76) ~[libuc-core.jar:?] at com.vmware.nsx.management.common.api.FacadeInterceptorAspect.annotatedMethod(FacadeInterceptorAspect.java:77) ~[libmp_common.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl.getFeedbackSummary(MigrationFacadeImpl.java:374) ~[libmc-core.jar:?] at com.vmware.nsxapi.migration.controller.MigrationController.getFeedbackSummary(MigrationController.java:551) ~[libmc-api.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_422] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_422] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_422] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_422] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.3.34.jar:5.3.34] at javax.servlet.http.HttpServlet.service(HttpServlet.java:529) ~[servlet-api.jar:?] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.34.jar:5.3.34] at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) ~[servlet-api.jar:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-websocket.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) ~[spring-web-5.3.34.jar:5.3.34] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[catalina.jar:9.0.91] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) ~[catalina.jar:9.0.91] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[catalina.jar:9.0.91] at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[catalina.jar:9.0.91] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346) ~[catalina.jar:9.0.91] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:388) ~[tomcat-coyote.jar:9.0.91] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-coyote.jar:9.0.91] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:936) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[tomcat-util.jar:9.0.91] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.91] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-util.jar:9.0.91]at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_422]2025-04-16T15:47:48.149Z INFO http-nio-127.0.0.1-7450-exec-2 AuditingServiceImpl 589938 - [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="migration-coordinator"] UserName="admin", ModuleName="Migration", Operation="GetFeedbackSummary", Operation status="failure", New value=[{"network_layer":"L3_L7","page_size":250}]2025-04-16T15:47:48.149Z ERROR http-nio-127.0.0.1-7450-exec-2 NsxBaseRestController 589938 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP100" level="ERROR" subcomp="migration-coordinator"]java.lang.NullPointerException: null at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackRequestFile(V2tFeedbackServiceImpl.java:458) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackRequests(V2tFeedbackServiceImpl.java:102) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.service.V2tFeedbackServiceImpl.getFeedbackSummary(V2tFeedbackServiceImpl.java:200) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl.getFeedbackSummary_aroundBody24(MigrationFacadeImpl.java:375) ~[libmc-core.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl$AjcClosure25.run(MigrationFacadeImpl.java:1) ~[libmc-core.jar:?] at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.9.0.jar:1.9.0] at com.vmware.nsx.management.upgrade.facade.FacadeInterceptorHelperImpl.proceed(FacadeInterceptorHelperImpl.java:76) ~[libuc-core.jar:?] at com.vmware.nsx.management.common.api.FacadeInterceptorAspect.annotatedMethod(FacadeInterceptorAspect.java:77) ~[libmp_common.jar:?] at com.vmware.nsx.management.migration.facade.MigrationFacadeImpl.getFeedbackSummary(MigrationFacadeImpl.java:374) ~[libmc-core.jar:?] at com.vmware.nsxapi.migration.controller.MigrationController.getFeedbackSummary(MigrationController.java:551) ~[libmc-api.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_422] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_422] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_422] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_422] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:903) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:809) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.3.34.jar:5.3.34] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.3.34.jar:5.3.34] at javax.servlet.http.HttpServlet.service(HttpServlet.java:529) ~[servlet-api.jar:?] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.34.jar:5.3.34] at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) ~[servlet-api.jar:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-websocket.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186) ~[spring-security-web-5.6.12.jar:5.6.12] at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) ~[spring-web-5.3.34.jar:5.3.34] at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) ~[spring-web-5.3.34.jar:5.3.34] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:168) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[catalina.jar:9.0.91] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130) ~[catalina.jar:9.0.91] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[catalina.jar:9.0.91] at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660) ~[catalina.jar:9.0.91] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[catalina.jar:9.0.91] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:346) ~[catalina.jar:9.0.91] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:388) ~[tomcat-coyote.jar:9.0.91] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-coyote.jar:9.0.91] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:936) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-coyote.jar:9.0.91] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[tomcat-util.jar:9.0.91] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[tomcat-util.jar:9.0.91] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[tomcat-util.jar:9.0.91] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_422]2025-04-16T15:47:48.150Z INFO http-nio-127.0.0.1-7450-exec-2 NsxBaseRestController 589938 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="migration-coordinator"] UserName:'admin' ModuleName:'common-services' Operation:'GET@/api/v1/migration/feedback-summary' Operation status: 'failure' Error: General error has occurred.
localhost_access_log.txt:11583:2025-04-14T23:37:04.639Z 127.0.0.1 - - "GET /migration-coordinator/api/v1/migration/feedback-summary?page_size=250&network_layer=L3_L7 HTTP/1.1" 500 169localhost_access_log.txt:11604:2025-04-15T00:16:51.516Z 127.0.0.1 - - "GET /migration-coordinator/api/v1/migration/feedback-summary?page_size=250&network_layer=L3_L7 HTTP/1.1" 500 169
var/log/migration-coordinator/v2t/collection.log:1451:Failed to GET https://192.168.###.##/api/2.0/services/securitygroup/vm-###### with status: 404 and reason: <?xml version="1.0" encoding="UTF-8"?> <error><errorCode>202</errorCode><details>The requested object : vm-###### could not be found. Object identifiers are case sensitive.</details><moduleName>core-services</moduleName></error>
This is a Day-1 issue since the security group plugin was introduced. However, this issue happens only when feedback file is very large in size; I.E. there are many recommendations to be either skipped or resolved.
V2T migrations are end of life from the VCF 9.0 Magnus and later therefore no permanent fix will be checked-in to the code.
***Manual Workaround***
1.) Take a backup of the NSX-T managers and of the two files to be rectified via 'cp': /opt/vmware/migration-coordinator-tomcat/bin/v2t/config-migrator/plugins/securitygroup_plugin.py & /var/log/migration-coordinator/feedback_request_l3_l7.json
2.) Stop the migration-coordinator service with the following command from the NSX-T manager admin shell: 'stop service migration-coordinator'
2.) Placeholders within the feedback_request_l3_l7.json file must be removed to ensure that the size of the file is acceptable. In the case of one instance of this issue the removal of placeholders decreased the file size from 93MB to 800KB. to remove said placeholders a 'jq' command can be run as shown below. JQ is a command-line JSON processing tool which is a good solution for dealing with machine-readable data formats.
jq 'map(if .message == "Security Groups with VMs/vNICs which are not on prepared hosts" then .details = "In NSX-T it is not supported to use SGs with VMs/vNICs not on prepared hosts.To proceed for the migration and not break existing security rules, migration coordinator will add an IP address expression for those VMs/vNICs in every such SG. Do you accept?" else . end)' feedback_request_l3_l7.json
3.) Ensure that file permissions and user groups of feedback_request_l3_l7.json is same after update. (the file in question is located here: /var/log/migration-coordinator/feedback_request_l3_l7.json)
4.) update the message in the "securitygroup_plugin.py" python script to remove the placeholders (to not fail with same issue the next time when migration is run). This file is located in /opt/vmware/migration-coordinator-tomcat/bin/v2t/config-migrator/plugins/securitygroup_plugin.py
-> Update 'detail_msg' declaration in the script to the below values:
detail_msg = "In NSX-T it is not supported to use SGs with VMs/vNICs not on " \
"prepared hosts. To proceed for the migration and not " \
"break existing security rules, migration coordinator will add an IP " \
"Address expression for those VMs/vNICs in every such SG. " \
"Do you accept?"
5.) Ensure that file permissions and user groups of /opt/vmware/migration-coordinator-tomcat/bin/v2t/config-migrator/plugins/securitygroup_plugin.py are the same after the update above.
6.) Start the migration-coordinator service from the NSX-T manager admin shell: 'start service migration-coordinator'
After the steps above the V2T migration wizard will load the recommendations properly and allow user to continue in rectifying the configuration that will be applied to NSX-T.