Minio VM start failing with error "Invalid license key" in Tanzu Platform for cloud foundry

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

Minio server started to fail after doing "apply change" on opsmanager.

Inside minio VM, minio-server process was in "not monitored" state. While checking the minio-server logs, we found below error lines. These logs were just giving connection refuse from other minio peer node.

PI: SYSTEM.gridTime: 00:17:01 UTC 03/25/2025DeploymentID: baa65973-24db-47e9-b733-9a9375847f0fError: grid: https://X.X.X.X:443 re-connecting to https://X.X.X.X:443: dial tcp X.X.X.X:443: connect: connection refused (*net.OpError) Sleeping 1.751s (3) (*fmt.wrapError)       6: internal/logger/logonce.go:118:logger.(*logOnceType).logOnceIf()       5: internal/logger/logonce.go:149:logger.LogOnceIf()       4: internal/grid/connection.go:59:grid.gridLogOnceIf()       3: internal/grid/connection.go:672:grid.(*Connection).connect.func1()       2: internal/grid/connection.go:678:grid.(*Connection).connect()       1: internal/grid/connection.go:275:grid.newConnection.func3()

On further checking, got below error in "bosh_dns_health.stdout.log". Here it is complaining for minio license key. 

Monitor] 2025-03-25T00:00:04.872617804Z WARN - Error occurred executing '/var/vcap/jobs/minio-server/bin/dns/healthy': Running command: '/var/vcap/jobs/minio-server/bin/dns/healthy', stdout: 'Invalid license key: failed to verify license: exp not satisfied', stderr: '': exit status 1[Monitor] 2025-03-25T00:00:04.872668567Z INFO - Status changed from running to failing

Environment

Minio Internal Blobstore for VMware Tanzu v1.x

Cause

 Found the license key applied on Minio tile was expired.

Resolution

Fetch the new Minio license key, apply it over tile and do the "apply change".