Apply Changes fails with "invalid_grant" and "Invalid JWT Signature" error

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

This error can show up if your BOSH director is using a GCS Blobstore. The full error message may look like this:

Preparing package bosh-dns: Fetching package blob: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-gcs cli: Running command: 'bosh-blobstore-gcs -c /var/vcap/bosh/etc/blobstore-gcs.json get <BLOB_GUID> /var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get#######', stdout: '', stderr: '2025/12/14 13:32:58 performing operation get: Get "https://storage.googleapis.com/blobstore-example/<BLOB_GUID>": oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"invalid_grant","error_description":"Invalid JWT Signature."}

Environment

BOSH director using a GCS Blobstore

Cause

The GCS blobstore configuration might be incorrect or the Service Account Key might have expired

Resolution

Verify the GCS blobstore configuration is correct under the BOSH Director Tile -> Director Config -> Blobstore Location. 

If the configuration is correct, check if the GCS service account key is expired and update if needed. Please refer to the Google IAM documentation to check the keys

To update the service key, you can run the "Recreate All Service Instances" errand, either in the tile or from the bosh command line.