The ( NULL byte in string ) error in CAPI logs.

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

We are only aware of a single incident where a 3rd party observability tool experienced issues querying applications in TAS.

Troubleshooting has identified that the issue was caused by an app environment variable.
We believe that a non-UTF character was added and it caused the CAPI query to fail during JSON parsing.

Returning the below error:

Request failed: 500: {\"error_code\"=>\"UnknownError\", \"description\"=>\"An unknown error occurred.\", \"code\"=>10001, \"test_mode_info\"=>{\"description\"=>\"NULL byte in string (after 

Environment

Product Version: 2.11

Resolution

With the use of cf curl we can identify the App with the offending ENV variable.
If the below curl fails to return the data on one of the 50 apps retrieved by this query, it means that the offending app ENV variable was being queried.

You would need to first increase page=1 value until you encounter the error. Once the error is encountered you need to decrease results-per-page=50 value to identify the app.

cf curl '/v2/apps?inline-relations-depth=0&order-direction=asc&page=44&results-per-page=50'

Example:
Notice the number of page=88 and results-per-page=25 re-adjusted in each attempt.

cf curl '/v2/apps?inline-relations-depth=0&order-direction=asc&page=88&results-per-page=25'

cf curl '/v2/apps?inline-relations-depth=0&order-direction=asc&page=176&results-per-page=13'

Eventually, isolate the App with the offending ENV variable by using the below query where the results-per-page=1.

cf curl '/v2/apps?order-direction=asc&page=269&results-per-page=1'

To validate the app, use the below curl command with the App GUID retrieved from the above command and query the App ENV variable.

cf curl /v3/apps/f2966104-30ea-46bc-8214-3c2966397155/environment_variables

 

Resolution:

Delete and redeploy the application to remove the offending App Environment variable.