The smoke_tests_windows errand keeps failing with error "Response exceeded maximum allowed length"

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

During an "Apply Changes" or a manual run-errand, the smoke_tests_windows errand keeps failing with error "Response exceeded maximum allowed length".

This issue was seen in a Windows Isolated Segment deployment. 

In the bosh agent logs, the smoke test was actually failing with the error "Unable to make an HTTP connection to the first app instance" and with HTTP response code 400.  The errand retried the connection many times but gave the same error output in every try, and due to that the volume of the output was too much to pass back to the Director via NATS, hence the error "Response exceeded maximum allowed length".

It appears that the error "Unable to make an HTTP connection to the first app instance" was due to the smoke-test app route having a domain that was being used in the TAS deployment.  However, the apps with this domain will have an ingress through the standard (non-isolated) gorouters in the TAS deployment, but the deployment where the smoke test was failing was set up for the Windows isolation segment, which means that the ingress should be through the isolated gorouters to be successful.  

We also saw in the TAS tile settings that the Gorouter rejects requests for all isolation segments (Settings->Networking->Requests to isolation segments->Reject requests for all isolation segements).  

In the Windows tile settings, the Smoke Test location setting is "A temporary space within the system org", which is the default too.  This uses the default app domain when deploying the smoke test app.

Environment

Elastic Application Runtime Windows

Cause

The Windows tile's Smoke Test location setting was set to "A temporary space within the system org", which deploys the smoke test app with the apps domain value configured for the TAS deployment.  However, the TAS tile settings (Networking) was configured to have the Gorouters reject requests for isolation segments.

Resolution

There are two options to work around this issue.

option 1:

• In the TAS tile settings "Networking->Requests to isolation segments->Reject requests for all isolation segements", set this to "Accept requests for all isolation segments", if your policy allows this.

option 2:

• In the Windows tile, set the Smoke Test location to a specified domain, org and space.  Use the domain that is being used in the isolated segment.  Make sure that DNS is configured to allow the smoke test app route to be resolvable to the load balancer for the isolation segment.