Unable to backup MySQL Instance in DSM 9.0.1

Products

• VMware Data Services Manager for VCF

Issue/Introduction

• Schedule backups of MySQL fail with below error
  

  d6894030-2f25-4cfa-b309-290fc04bfb59/backups/2025/11/18/20251118T235912-7xv584-backup.xb") (code=Unknown): operation error S3: PutObject, https response error StatusCode: 400, RequestID: 1763536005140557, HostID: 12271399, api error InvalidArgument: Invalid Argument

• No issues observed with Postgres SQL backups.
• Issue is seen in DSM 9.0.1 version.

Environment

Data Service Manager 9.0.1

Cause

Starting with DSM version 9.0.1, MySQL uses a newer version of the AWS S3 SDK that automatically enables data integrity checks for S3 uploads. This feature uses a specific S3 protocol mechanism called "STREAMING-UNSIGNED-PAYLOAD-TRAILER" to calculate checksums during uploads.

Some S3-compatible storage services (such as AVM Cloud) do not yet support this protocol mechanism. When the S3 client attempts to upload backups to these services, the upload fails with a 400 InvalidArgument error or similar error because the storage service cannot process the checksum-related headers that the newer AWS SDK sends by default.

Resolution

Workaround:

There are two things that can be done:

A) First, Contact the S3-compatible storage vendor (AVM Cloud in this case) and ask if and when they plan to support the STREAMING-UNSIGNED-PAYLOAD-TRAILER protocol mechanism. This is a standard AWS S3 feature documented in the AWS S3 API documentation, and many vendors are working on adding support for it.

B) You can switch to data plane version 9.0.0.0 for MySQL clusters only, which uses an older AWS SDK version that does not enable automatic checksums by default. 9.0.0.0 version of MysQL can be used in DSM 9.0.1 deployment. This version is also compatible with S3 services that do not support the streaming unsigned payload trailer mechanism. To do that there 2 steps:

Step 1. First download the "DSM Update Bundle" for version 9.0.0.0 from download Portal:

• Uploading VMware Data Services Manager Release Bundle

Step 2. Follow instructions to add the new bundle to existing Data Services Manager

• Upload the VMware Data Services Manager Update Bundle Using the VMware Data Services Manager UI

Permanent Fix:

In the next release (version 9.0.2), we will introduce a new configuration option checksumCalculationUnsupported in the BackupLocation resource. This option will allow users to disable the automatic checksum calculation for S3 uploads, making the data service compatible with S3 services that do not support the streaming unsigned payload trailer mechanism.