How To Fix DB Migration Error on Manager Due To Network Proxy Changes or Failures
search cancel

How To Fix DB Migration Error on Manager Due To Network Proxy Changes or Failures

book

Article ID: 333617

calendar_today

Updated On:

Products

VMware

Issue/Introduction

A db migration error can occur while attempting to make configuration changes on a Manager when it is using a Proxy server and that proxy server is misconfigured, fails or blocks connections from the Manager to the DC.

Symptoms:

If the lastline_apply_config configuration change operation fails to complete because of a db migration error after changing the Proxy server setting using lastline_setup because the proxy does not allow the Manager access to VMware Data Center (DC) for any reason, you will need to follow the steps in the Workaround section below to resolve this error.

In the example below, a new proxy was configured on the Manager with a Proxy server that had enabled Proxy Authentication required. Our Manager does not support Proxy Authentication so the Proxy blocked the attempts by the Manager to communicate with our DC. The lastline_apply_config operation did not complete successfully. We applied the workaround to fix the configuration error and the Proxy Server configuration was changed to not require Proxy Authentication.

Example Proxy Authentication Error:

The error  we saw in the lastline_apply_config log:

2022-07-29 17:47:40,606 - INFO - Pulling db-customer-migration ... error
2022-07-29 17:47:40,607 - INFO -
2022-07-29 17:47:40,607 - INFO - ERROR: for db-customer-migration b'Get https://update.lastline.com/v2/: Proxy Authentication Required'
2022-07-29 17:47:40,607 - INFO - Get https://update.lastline.com/v2/: Proxy Authentication Required
2022-07-29 17:47:40,607 - INFO - WARNING:docker_compose.pull:Invoking docker-compose 'pull' for service db-customer-migration (project db_migrations) failed: Command '['sudo', '--set-home', '--preserve-env', '--user', 'docker_user', '--', '/usr/bin/docker-compose', '--file', '/usr/share/appliance-config/templates/db_migrations/docker-compose-db-migrations.yml', '--project-name', 'db_migrations', 'pull', 'db-customer-migration']' returned non-zero exit status 1
2022-07-29 17:47:40,607 - INFO - ERROR:appliance_config.services.db_migrations:Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed
2022-07-29 17:47:40,608 - INFO - ERROR:appliance_config.services.db_migrations:Could not run service db-customer-migration: Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed. Saving tiller configuration to /tmp/run_db_customer_migration_v3U2Ge
2022-07-29 17:47:40,608 - INFO - INFO:appliance_config.services.db_migrations:Cleaning up containers..
2022-07-29 17:47:40,608 - INFO - The DB_ANALYSIS_IMAGE variable is not set. Defaulting to a blank string.
2022-07-29 17:47:40,608 - INFO - The DB_ANALYSISSHARD_IMAGE variable is not set. Defaulting to a blank string.
2022-07-29 17:47:40,608 - INFO - The DB_ANALYST_IMAGE variable is not set. Defaulting to a blank string.
2022-07-29 17:47:40,608 - INFO - INFO:appliance_config.services.db_migrations:Cleaning up completed!
2022-07-29 17:47:40,609 - INFO - ERROR:__main__:Could not apply migration Could not run service db-customer-migration: Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed



Cause

If you replace an existing proxy configuration with a new proxy and the new proxy blocks the Manager appliance from communicating with our DC, the new proxy configuration change will fail because Puppet can not run the db migration operation that requires communication to our DC.

Resolution



Workaround:
The workaround for this error is to run lastline_apply_config with the the option -skip-pre-puppet-db-migrations  to have it skip the db migration operation to allow puppet to run and update to the new proxy configuration.
 
lastline_apply_config --skip-pre-puppet-db-migrations

Next, run the lastline_apply_config again to have it run the db migration operation.
lastline_apply_config 

If there are any services reporting down in the Portal UI, use the Retrigger configuration menu option in the Quick Menu drop down on the right side of the UI for that appliance. This will update the service status in the UI.

Additional Information

Impact/Risks:
Manager remains in a configuration error state until the workaround is applied.