Troubleshooting NSX API Calls
Article ID: 379535
Updated On:
Products
Issue/Introduction
NSX-T uses REST API calls, both internally and externally to CRUD (Create, Read, Update, Delete) resources.
This guide will outline the usage of these API call, best practices and troubleshooting steps.
Environment
VMware NSX
Resolution
The API guide is structured similar to the NSX-T UI and features, with headings:
- API Usage - details how to use the API's with NSX-T.
- Federation - API's which can be used for Federated environments.
- Management API - This refers to the management mode of NSX-T, the management API have been deprecated and will be removed in a future version since NSX-T 3.2, see VMware NSX-T Data Center 3.2 Release Notes
- NSX Application Platform (NAPP) - These are API's which can be used for the NAPP feature.
- Policy - These API's are now the preferred and supported option when working with NSX-T and are used to configure NSX-T in Policy mode.
- Search - Used to build queries used for searching NSX-T
- System - These API's are used mainly for system administration, such as backup/restore, user management, certificates, etc.
A note on the difference between Manager and Policy mode, Policy mode is the desired state, it send the configuration which is desired to Manager mode and Manager mode then implements the desired configuration. Manager API's start with /api and Policy mode API's start with /policy/api.
NSX-T has some limitation in regards the number of API call which it can handle, these are outlined at the start of the API guides for each version.
In general they are split into 3 categories:
- per-client rate limit, in requests per second. If a client makes more requests than this limit in one second, the API server will refuse to service the API request and will return an HTTP 429 Too Many Requests Error. By default, this limit is 100 requests per second.
- per-client concurrency limit. This is the maximum number of outstanding requests that a client can have. For example, a client can open multiple connections to NSX and submit operations on each connection. When this limit is exceeded, the server returns a 503 Server Unavailable error to the client. By default, this limit is 40 concurrent requests.
- An overall maximum number of concurrent requests. This is the maximum number of API requests that can be in process on the server. If the server is at this limit, additional requests will be refused and the HTTP error 503 Service Unavailable will be returned to the client. By default, this limit is 199 concurrent requests.
The HTTP response code(s) are sent back to the client making the API calls, in a well behaving client, it should note the response and back-off and retry later. In the NSX-T manager, these response code(s) are logged in the manager log: /var/log/proxy/reverse-proxy.log, each manager's proxy service is responsible for maintaining its client limits.
Logs in proxy.log look like:
- Global concurrency limit:
"Number of global active requests exceeded allowed maximum of {} {} times in last {} seconds" - Per-client concurrency limit:
"Number of client active requests exceeded allowed maximum of {} {} times in last {} seconds" - Per-client rate limit:
"Allowed client rate was exceeded {} times in last {} seconds. Allowed= {}, Current rate= {}, "
In NSX-T 4.0, the proxy was changed to envoy, local API calls now reside in /var/log/proxy/envoy_access_log.txt, prior to this change, local API calls where logged in /var/log/proxy/localhost_access_log.txt. The log /var/log/proxy/reverse-proxy.log is used on the API leader to proxy connections to other managers.
Known Issues:
REST API calls using a vIDM user fail in NSX-T
NSX-T slow API/UI response time
Logs show password in clear text from API calls made to the NSX-T Manager
Unable to access NSX-T UI and API failure following a change in Manager Node certificates
During an NSX-T Manager restore from backup, HTTPS service fails to start
Additional Information
Feedback
