Error "failed to delete NSX-T cluster network" in NSX-T OSB-Proxy

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

TKGi cluster is stuck "in progress" state. This can happen when cluster is being created or deleted.
The Master and Worker VMs have been created successfully.

NSX-T OSB-Proxy throws error:

"error":"failed to delete NSX-T cluster network pks-########-8b8b-4100-91fe-############: json: cannot unmarshal string into Go struct field RelatedAPIError.error_code of type int64","session":"3

The OSB Proxy is failing to create/delete the network during the TKGi creation/deletion.

Environment

Product Version: >=1.8

Resolution

To determine the reason why, the NSX-T Support team will need to be engaged and NSX-T logs will need to be reviewed.

However, during this network creation/deletion process, the NSX-T firewall sections are listed and it has been observed that if there is a corrupt or invalid firewall section, it will fail to list them and the unmarshal error above is observed on OSB Proxy. 

Review NSX-T logs and check for any issues listing firewall sections:
"POST /api/v1/firewall/sections/<GUID>?action=list_with_rules HTTP/1.1" 500 171 11 1"

The corrupt or invalid NSX-T firewall section can be deleted using NSX-T API.