Production Docker Portal Down (Stuck on Checking to see if the Db schema update is needed)

Products

• CA API Developer Portal

Issue/Introduction

Trying to offbox the database on Production Portal from local Postgres to External mysql.

However, when trying to edit the portal.conf file and restart portal wtith:   sudo ./portal.sh

the portal would be stuck for over 20 minutes checking the schema and would not start.

example below :

bash-4.2# ./portal.sh

Creating network portal_db

Creating service portal_db-upgrade

Creating service portal_db-upgrade-rbac

Checking to see if DB schema update is needed.......

If trying to revert the configuration back to local postgres, and the portal.sh script is stuck on the schema check as well.

 

Environment

Release : 5.0

Component : API PORTAL

Resolution

1. From linux terminal run :

  docker stack rm portal  (to cleanup all container images running when failed the command below)

  sudo ./portal.sh stays running without finishing.. "Checking to see if the Db schema update is needed"

2. open a Second linux terminal to run :

  watch "docker service ls"

  to show how much images are running while portal.sh is running in parallel 

3. Opena 3rd linux terminal , run :  docker service ls

you should see Images Names , like :

    portal_db-upgrade

    portal_db-upgrade-rbac

    portal_portaldb

 
4. Observed there are 3 images for portal currently  but not observed portal_slave ones (master and slave are not there).. means something is Wrong to start.

4. Run :  docker service logs -f portal_portaldb , to see more info
 
5. executed :    docker stack rm portal 
  to clean all the current containers and try again with portal.sh which did not started.
 
6. Alternative to start portal again is, copying the images folder to  /apim-portal-5.0-final folder
   currently there is an /image folder but empty.
 
7. Extracted the /images folder from the package GEN0000000003764.tar.gz and copied into  apim-portal-5.0-final folder

    sudo cp -R images/  /opt/apim-portal-5.0-final 

8. Then run ./portal.sh again and portal container should start healthy

9. Also,  Database migration completed successfully. 

Additional Information

To download Portal 5.x

example :

https://support.broadcom.com/download-center/product-files.html?sku_code=APIXSF99000&release=5.0&gen_level=0000&language=EN

API Developer Portal Installer Offline 5.0

(GEN0000000003764.tar.gz)