Smoke Test Errand Fails on Isolation Segment Tile with "Unable to print logs for app"

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

During the installation or upgrade of the VMware Tanzu Isolation Segment tile, the smoke_tests_isolation errand fails during the Apply Changes phase. While the errand fails, manually pushing an application (cf push) to the isolated segment Diego cell completes successfully.

When reviewing the errand logs, the test is shown to successfully push the application, start it, and verify its health, but fails and times out when attempting to stream the logs via cf logs.

The following error signature can be found in the Isolation Segment errand logs:

No future change is possible.  Bailing out early after 0.176s.
Unable to print logs for app.
Got stuck at:
    Retrieving logs for app SMOKES-APP-661fbcd4-29f0 in org system / space CF_SMOKE_TEST_SPACE as smoke_tests...

Environment

Tanzu Platform Cloud Foundry (TPCF) / Elastic Application Runtime (EAR)

Tanzu Isolation Segment

 

Cause

The smoke test errand validates the environment by pushing a test application and immediately attempting to retrieve its logs using the cf logs --recent command. If the Loggregator subsystem on the main TPCF/EAR foundation is under-resourced (experiencing CPU or memory contention), log delivery can be delayed.

Because the smoke test enforces a strict timeout threshold for log retrieval, any delay from the doppler or log-cache components will cause the errand to time out and fail with the Unable to print logs for app error, even if the application itself is running perfectly fine on the Diego cell.

Resolution

To resolve this issue, you must scale up the resources for the Loggregator components (doppler and log-cache) within the main TPCF/EAR tile to ensure they can process and serve logs without delay.

1. Log in to the Ops Manager UI.

2. Click on the VMware Tanzu Application Service (TAS) tile.

3. Navigate to the Resource Config pane.

4. Locate the Log Cache (log-cache) and Doppler Server (doppler) jobs.

5. Increase the VM Type (CPU/Memory) and/or the number of Instances for these jobs. (Example: In one environment, scaling log-cache from 1 instance to 2 instances with higher RAM, and doppler from 1 instance to 2 instances resolved the issue).

6. Click Save.

7. Navigate back to the Installation Dashboard and click Review Pending Changes.

8. Select only the TAS tile and click Apply Changes to deploy the scaled-up logging VMs.

9. Once the TPCF/EAR deployment completes successfully, select the Isolation Segment tile and run Apply Changes to re-trigger the smoke test errand. It should now pass.

 

Additional Information

 

• Simply recreating the VMs (bosh recreate) will generally not resolve this issue if the root cause is a lack of allocated CPU/Memory resources for the logging components.

• Ensure that the underlying infrastructure (e.g., ESXi hosts) has sufficient available physical resources to accommodate the scaled-up VMs, as underlying host resource contention can also bottleneck log processing.