Upgrading service instance to Tanzu RabbitMQ on Cloud Foundry 10.0 hits error failed_to_deny_deprecated_features
Article ID: 387844
Updated On:
Products
Issue/Introduction
After Tanzu RabbitMQ on Cloud Foundry tile is upgraded to release 10.0, the upgrade-all-service-instances errand might fail with updating some service instances. For example,
[upgrade-all-service-instances] 2025/02/06 23:09:39.160552 [upgrade-all] FINISHED PROCESSING Status: FAILED; Summary: Number of successful operations: 0; Number of skipped operations: 25; Number of service instance orphans detected: 0; Number of deleted instances before operation could happen: 0; Number of busy instances which could not be processed: 0; Number of service instances that failed to process: 1 [7a27d937-xxxx-xxxx-xxxx-6145e2ca5394]
[upgrade-all-service-instances] 2025/02/06 23:09:39.160567 canaries didn't process successfully: [7a27d937-xxx-xxxx-xxxx-6145e2ca5394] Operation failed: bosh task id 0:When checking BOSH logs for deploying instance 7a27d937-xxxx-xxxx-xxxx-6145e2ca5394, it says failure in starting rabbitmq-server job.
Environment
Tanzu RabbitMQ on Cloud Foundry 10.0
Cause
When reviewing rabbitmq-server job logs, could see failed_to_deny_deprecated_features error in log file startup_stderr.log
# startup_stderr.log
BOOT FAILED
===========
Error during startup: {error,
{failed_to_deny_deprecated_features,
[classic_queue_mirroring]}}
Runtime terminating during boot (terminating)
Crash dump is being written to: /var/vcap/sys/log/rabbitmq-server/erl_crash.dump...done
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
[os_mon] memory supervisor port (memsup): Erlang has closed
And similar logs in file "[email protected]"
2025-02-07 00:31:42.326740+00:00 [notice] <0.372.0> Feature flags: attempt to enable `classic_queue_mirroring`...
2025-02-07 00:31:42.380813+00:00 [info] <14770.9574.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2025-02-07 00:31:42.380969+00:00 [info] <14770.9574.0> Successfully synced tables from a peer
2025-02-07 00:31:42.381082+00:00 [error] <14770.9574.0> Deprecated features: `classic_queue_mirroring`: can't deny deprecated feature because it is actively used
2025-02-07 00:31:42.379837+00:00 [info] <14771.10590.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2025-02-07 00:31:42.380022+00:00 [info] <14771.10590.0> Successfully synced tables from a peer
2025-02-07 00:31:42.380113+00:00 [error] <14771.10590.0> Deprecated features: `classic_queue_mirroring`: can't deny deprecated feature because it is actively used
2025-02-07 00:31:42.437791+00:00 [error] <0.372.0> Feature flags: failed to enable `classic_queue_mirroring`: {error,
2025-02-07 00:31:42.437791+00:00 [error] <0.372.0> {failed_to_deny_deprecated_features,
2025-02-07 00:31:42.437791+00:00 [error] <0.372.0> [classic_queue_mirroring]}}
2025-02-07 00:31:42.444447+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0>
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0> BOOT FAILED
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0> ===========
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0> Error during startup: {error,
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0> {failed_to_deny_deprecated_features,
2025-02-07 00:31:42.444617+00:00 [error] <0.254.0> [classic_queue_mirroring]}}The error indicates the deprecated Classic Queue Mirroring feature is enabled for the problematic RabbitMQ instance, which is now removed in the new release 10.0. However, since the instance is still actively using this deprecated feature and can't be denied, the instance would refuse to boot up.
Resolution
As it's stated in this KB article, warning will be shown on the instance management dashboard since RabbitMQ version 3.13.x. And the deprecated Classic Queue Mirroring feature is removed from version 10.0 (OSS 4.x) according to RabbitMQ document, which means this feature is not available after upgrading to version 10.0. Therefore, the RabbitMQ service instance should be updated not to use the Classic Queue Mirroring feature in order to address the issue. Or migrate existing mirrored classic queue to quorum queue (see reference: https://www.rabbitmq.com/docs/3.13/migrate-mcq-to-qq).
Feedback
