Proxy gateway stays disconnected after Portal upgrade to 5.2.x

Products

CA API Gateway CA API Developer Portal

Issue/Introduction

After upgrading API portal to 5.2.x the proxy gateway stays disconnected in the proxy details .

Restarting the portal deployer service on the gateway does not resolve the issue and the logs show the deployer service is connecting to the broker without errors.

Environment

Portal 5.2.x

Cause

The problem is caused by a old docker version 18.09.3 which is included in the Centos ova 3 image.

The rabbitmq container is showing an error on startup

On startup of the rabbitmq container it can not read the config from the attached volume

Line 24584: Feb 6 14:34:18 localhost 93929c16fa9f[2037]: #033[38;5;6mrabbitmq #033[38;5;5m13:34:18.83 #033[0m#033[38;5;2mINFO #033[0m ==> Initializing RabbitMQ...
Line 24585: Feb 6 14:34:18 localhost 93929c16fa9f[2037]: ls: cannot access '/bitnami/rabbitmq/conf': Operation not permitted
Line 24586: Feb 6 14:34:18 localhost 93929c16fa9f[2037]: ls: cannot access '/opt/bitnami/rabbitmq/.rabbitmq/mnesia': Operation not permitted
Line 24587: Feb 6 14:34:18 localhost 93929c16fa9f[2037]: #033[38;5;6mrabbitmq #033[38;5;5m13:34:18.89 #033[0m#033[38;5;2mINFO #033[0m ==> Starting RabbitMQ in background...

Which throws the following error when we try to get the proxy status

Line 5896: Feb 6 15:10:20 localhost 93929c16fa9f[2037]: #033[38;5;160m2024-02-06 14:10:20.885051+00:00 [error] <0.1333.0> operation basic.publish caused a channel exception not_found: no exchange 'portal-external' in vhost '/'#033[0m

Resolution

Upgrade docker on the Centos ova 3 image or migrate to the newer Centos ova 4 or Debian OVA image .

https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-developer-portal/5-2/install-configure-and-upgrade/install-portal-on-docker-swarm/prepare-your-environment.html

To update the current centos ova 3 to the latest docker version and apply latest OS updates follow the following steps :

Make sure you can pull images from the internet using Yum and take backup or snapshot of the portal server and databases.

Stop portal with "docker stack rm portal"
Remove the versionlock for docker "yum versionlock clear"
Update the image with "yum update" this take some time about 400 packages

After this is finished , "yum update" updated Docker to version 25.x , adding overlay2 configurations in /etc/docker/daemon.json.

However, the backing filesystem on OVA 3 does not support this added configuration.

Edit the daemon.json file and remove the "storage-driver" and "storage-opts" lines, as illustrated in the following snippet:

{
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"log-driver": "journald",
"selinux-enabled": true
}

Restart the server .
Check if docker is started and which version "docker version"
Deploy portal again by running "portal.sh"
Check rabbitmq logs and proxy status it may be needed to restart the deployer service on the gateway once more .