Workload Domain deployment fails while creating NSX Conroller VMs
search cancel

Workload Domain deployment fails while creating NSX Conroller VMs

book

Article ID: 316760

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

Symptoms:
  • Workload Domain deployment fails while creating NSX Conroller VMs in VMware Cloud Foundation 2.4
  • You see errors similar to the following in the /home/vrack/vrm/logs/evosddc-debug.log file on the SDDC Manager Controller virtual machine:
2018-03-12 22:44:03.221 [https-jsse-nio-8443-exec-63] DEBUG [com.vmware.vrack.vrm.ha.service.fsm.BringupFsmTasksSource] Loaded 2 FSM Workflows from http://localhost:9080/bringup-app/bringup/sddcs
2018-03-12 22:44:10.776 [pool-23-thread-2] DEBUG [com.vmware.vrack.vrm.nsx.NsxServiceImpl] Retry: create controller node
2018-03-12 22:44:10.776 [pool-23-thread-2] DEBUG [com.vmware.vrack.vrm.workflow.tasks.nsx.CreateNsxControllerTask] Progress message: Failed to deploy NSX controller: Node-3
2018-03-12 22:44:10.777 [pool-23-thread-2] DEBUG [com.vmware.vrack.vrm.workflow.tasks.nsx.CreateNsxControllerTask] <<< Failed to deploy NSX controller: Node-3 VDI-VDI-FullClone
2018-03-12 22:44:10.779 [pool-23-thread-2] ERROR [com.vmware.vrack.vrm.workflow.tasks.nsx.CreateNsxControllerTask] Exception creating NSX Controller
com.vmware.vrack.vrm.nsx.NsxException: Failed to create Controller node
        at com.vmware.vrack.vrm.nsx.NsxServiceImpl.createControllerCluster(NsxServiceImpl.java:590)
        at com.vmware.vrack.vrm.workflow.tasks.nsx.CreateNsxControllerTask.doInNsx(CreateNsxControllerTask.java:77)
        at com.vmware.vrack.vrm.workflow.tasks.AbstractNsxTask.run(AbstractNsxTask.java:45)
        at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.executeTask(SimpleWorkflowEngine.java:550)
        at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.runSingleTask(SimpleWorkflowEngine.java:501)
        at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.access$1000(SimpleWorkflowEngine.java:47)
        at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine$2.run(SimpleWorkflowEngine.java:311)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-03-12 22:44:12.874 [PSM-03122018104412331] DEBUG [com.vmware.vrack.vrm.prm.dbservice.service.PrmInventoryServiceImpl] Inside PrmInventoryServiceImpl.getRackInfo() with rackId: 609141d7-773e-4ccc-8db5-46f9fe4353bb
2018-03-12 22:44:13.135 [PSM-03122018104412331] DEBUG [com.vmware.vrack.vrm.prm.dbservice.service.PrmInventoryServiceImpl] Inside PrmInventoryServiceImpl.getRackInfo() with rackId: 609141d7-773e-4ccc-8db5-46f9fe4353bb
2018-03-12 22:44:13.378 [https-jsse-nio-8443-exec-5] DEBUG [com.vmware.vrack.vrm.security.websso.util.RequestAuthorizationUtility] isRequestPreAuthorized url /vrm-ui/api/1.0/events/counts ip 10.0.1.116
2018-03-12 22:44:13.379 [https-jsse-nio-8443-exec-5] DEBUG [com.vmware.vrack.vrm.security.websso.util.RequestAuthorizationUtility] Bringup status: true
2018-03-12 22:44:13.379 [https-jsse-nio-8443-exec-5] ERROR [com.vmware.vrack.vrm.security.websso.util.RequestAuthorizationUtility] Not all pre-authorization conditions met for URL /vrm-ui/api/1.0/events/counts called from 10.0.1.116
2018-03-12 22:44:13.380 [https-jsse-nio-8443-exec-5] DEBUG [com.vmware.vrack.vrm.security.websso.util.RequestAuthorizationUtility] URL /vrm-ui/api/1.0/events/counts is not in the basic-auth list for IP 10.0.1.116.
 
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Resolution

This is a known issue affecting VMware Cloud Foundation 2.4. There is currently no resolution.

Workaround:
To workaround this issue, delete the NSX Controller VM, per the following process:
  1. Launch a web browser with a REST client installed (postman, restlet, etc...) from a system with access to the VMware Cloud Foundation management nework.
  2. In the REST client, create a new PUT method against an endpoint similar to the following:
https://<NSX Manager IP address>/api/2.0/vdn/controller/controller-<#>?forceRemoval=False

Notes:
  • Enter the admin user password when prompted. The admin user password can be obtained by running the /home/vrack/bin/lookup-passwords command on the SDDC Manager Controller VM.
  • Replace <NSX Manager IP Address> with the IP address of the NSX Manager VM.
  • Replace <#> with the number of the NSX Controller to be deleted (1,2 or 3).
  • If the last controller is being removed, replace forceRemoval=False with forceRemoval=True in the previous PUT method.
The failed workload domain workflow can be retried at this point.

Powered by Wolken Software