How To Fix DB Migration Error on Manager Due To Network Proxy Changes or Failures
Article ID: 333617
Updated On:
Products
Issue/Introduction
Symptoms:
If the lastline_apply_config configuration change operation fails to complete because of a db migration error after changing the Proxy server setting using lastline_setup because the proxy does not allow the Manager access to VMware Data Center (DC) for any reason, you will need to follow the steps in the Workaround section below to resolve this error.
In the example below, a new proxy was configured on the Manager with a Proxy server that had enabled Proxy Authentication required. Our Manager does not support Proxy Authentication so the Proxy blocked the attempts by the Manager to communicate with our DC. The lastline_apply_config operation did not complete successfully. We applied the workaround to fix the configuration error and the Proxy Server configuration was changed to not require Proxy Authentication.
Example Proxy Authentication Error:
The error we saw in the lastline_apply_config log:
2022-07-29 17:47:40,606 - INFO - Pulling db-customer-migration ... error 2022-07-29 17:47:40,607 - INFO - 2022-07-29 17:47:40,607 - INFO - ERROR: for db-customer-migration b'Get https://update.lastline.com/v2/: Proxy Authentication Required' 2022-07-29 17:47:40,607 - INFO - Get https://update.lastline.com/v2/: Proxy Authentication Required 2022-07-29 17:47:40,607 - INFO - WARNING:docker_compose.pull:Invoking docker-compose 'pull' for service db-customer-migration (project db_migrations) failed: Command '['sudo', '--set-home', '--preserve-env', '--user', 'docker_user', '--', '/usr/bin/docker-compose', '--file', '/usr/share/appliance-config/templates/db_migrations/docker-compose-db-migrations.yml', '--project-name', 'db_migrations', 'pull', 'db-customer-migration']' returned non-zero exit status 1 2022-07-29 17:47:40,607 - INFO - ERROR:appliance_config.services.db_migrations:Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed 2022-07-29 17:47:40,608 - INFO - ERROR:appliance_config.services.db_migrations:Could not run service db-customer-migration: Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed. Saving tiller configuration to /tmp/run_db_customer_migration_v3U2Ge 2022-07-29 17:47:40,608 - INFO - INFO:appliance_config.services.db_migrations:Cleaning up containers.. 2022-07-29 17:47:40,608 - INFO - The DB_ANALYSIS_IMAGE variable is not set. Defaulting to a blank string. 2022-07-29 17:47:40,608 - INFO - The DB_ANALYSISSHARD_IMAGE variable is not set. Defaulting to a blank string. 2022-07-29 17:47:40,608 - INFO - The DB_ANALYST_IMAGE variable is not set. Defaulting to a blank string. 2022-07-29 17:47:40,608 - INFO - INFO:appliance_config.services.db_migrations:Cleaning up completed! 2022-07-29 17:47:40,609 - INFO - ERROR:__main__:Could not apply migration Could not run service db-customer-migration: Could not pull image for service db-customer-migration: Invoking docker-compose (pull) failed
Cause
Resolution
Workaround:
The workaround for this error is to run lastline_apply_config with the the option -skip-pre-puppet-db-migrations to have it skip the db migration operation to allow puppet to run and update to the new proxy configuration.
lastline_apply_config --skip-pre-puppet-db-migrations
Next, run the lastline_apply_config again to have it run the db migration operation.
lastline_apply_config
If there are any services reporting down in the Portal UI, use the Retrigger configuration menu option in the Quick Menu drop down on the right side of the UI for that appliance. This will update the service status in the UI.
Additional Information
Manager remains in a configuration error state until the workaround is applied.
Feedback
