VM creation on Azure fails in PKS with '#<NoMethodError: undefined method `[]' for nil:NilClass>'
Article ID: 298498
Updated On:
Products
VMware Tanzu Kubernetes Grid Integrated Edition
Issue/Introduction
Symptoms:
If a user has configured load balancer under PKS Tile > Resource Config > Load Balancer, and the load balancer configuration is missing a property in Azure, bosh VM creation tasks error out with the following message:
If a user has configured load balancer under PKS Tile > Resource Config > Load Balancer, and the load balancer configuration is missing a property in Azure, bosh VM creation tasks error out with the following message:
Task 24 | 02:17:58 | Creating missing vms: pivotal-container-service/aa0c97af-b6a9-48ee-8f97-52950e567132 (0) (00:00:16)
L Error: CPI error 'Bosh::Clouds::VMCreationFailed' with message '#<NoMethodError: undefined method `[]' for nil:NilClass>
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/lib/cloud/azure/restapi/azure_client.rb:1428:in `create_network_interface'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/lib/cloud/azure/vms/vm_manager_network.rb:194:in `block (2 levels) in _create_network_interfaces'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:24:in `block in execute'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/safe_task_executor.rb:19:in `execute'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/ivar.rb:170:in `safe_execute'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/future.rb:52:in `block in execute'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/var/vcap/data/packages/bosh_azure_cpi/cba89fb37c02859c989b5329188b2f395b827f24/vendor/bundle/ruby/2.4.0/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
Updating deployment:
Expected task '24' to succeed but state is 'error'
Exit code 1
Environment
Cause
The CPI error '
For example, this can happen if there was no backend pool configured. From the CPI source code, observe that while creating a network interface, CPI tries to get the backend address pool id. If the backend address pool returns NULL, the VM creation fails.
For more information on the CPI source code, refer to the following: https://github.com/cloudfoundry/bosh-azure-cpi-release/blob/master/src/bosh_azure_cpi/lib/cloud/azure/restapi/azure_client.rb#L1425-#L1433
Bosh::Clouds::VMCreationFailed' with message '#<NoMethodError: undefined method `[]' for nil:NilClass>' is cryptic and there is no way for someone debugging to determine from the logs what the root cause of the problem is.For example, this can happen if there was no backend pool configured. From the CPI source code, observe that while creating a network interface, CPI tries to get the backend address pool id. If the backend address pool returns NULL, the VM creation fails.
For more information on the CPI source code, refer to the following: https://github.com/cloudfoundry/bosh-azure-cpi-release/blob/master/src/bosh_azure_cpi/lib/cloud/azure/restapi/azure_client.rb#L1425-#L1433
Resolution
To fix this problem, Azure load balancer settings need to be verified. In this specific case, adding the missing VM to the backend pool under Azure load balancer will resolve the issue.
Feedback
Yes
No
Powered by
