Migration of vCenter is stuck at Exporting VCDB in Stage 2

Products

• VMware vCenter Server

Issue/Introduction

This KB helps to isolate the Migration of vCenter being Stuck at "Exporting VCDB".

Symptoms:
Migration from Windows vCenter to VCSA is Stuck at "Exporting VCDB" in Stage 2 and never fails, after 60 minutes, it will give an option to continue the export, however it will still not complete and be stuck at "Exporting VCDB"..

In the Windows vCenter in the Export Folder, we noticed that the export of VCDB file Size did not increase.

upgrade-export.log:

2020-04-13T13:05:35.109Z INFO __main__ Executing EXPORT phase: export
2020-04-13T13:05:35.598Z INFO upgrade_commands Start exporting data from vcenter1.komodo.ch
2020-04-13T13:05:36.798Z INFO upgrade_commands Export hasn't begun. Waiting..
2020-04-13T13:05:42.846Z INFO upgrade_commands Export hasn't begun. Waiting..
2020-04-13T13:41:08.512Z INFO status_reporter Question was asked from remote target: {'reply_file': 'c:\\users\\admin\\appdata\\local\\temp\\2\\6dc6ccde7d8c11ea9222005056812b6c.json', 'id': '6dc6ccde7d8c11ea9222005056812b6c', 'text': {'id': 'vcdb.wait.jobs.complete.question', 'translatable': 'Database job(s) did not complete for 2000 seconds.\nContinue to wait?\nClick Yes to wait, no to cancel upgrade.', 'localized': 'Database job(s) did not complete for 2000 seconds.\nContinue to wait?\nClick Yes to wait, no to cancel upgrade.'}, 'type': 'YES_NO', 'default': 0}

Export_com.vmware.vcdb_2020_04_13_14_29.log:

2020-04-13T12:31:11.974Z INFO service_manager Service vpxd reported status stopped. Expected status stopped
2020-04-13T12:31:11.974Z INFO service_manager Service vpxd: STOPPED
2020-04-13T12:31:11.983Z INFO vcdb.const Retrieving DB type...
2020-04-13T12:31:11.992Z INFO vcdb.const Retrived DB type: mssql
2020-04-13T12:31:11.993Z INFO vcdb.const User option to migrate tables: core
2020-04-13T12:31:12.5Z INFO vcdb.const VCDB: external
2020-04-13T12:31:12.5Z INFO vcdb.const Database is external. Migrate to VCSA 6.6.
2020-04-13T12:31:12.5Z INFO vcdb.const Retrieving DB type...
2020-04-13T12:31:12.9Z INFO vcdb.const Retrieved DB type: mssql
2020-04-13T12:31:12.16Z INFO vcdb.const Getting vCenter Server DSN...
2020-04-13T12:31:12.20Z INFO vcdb.const vCenter Server uses DSN: vCenter
2020-04-13T12:31:12.20Z INFO vcdb.const Retrieving DB user...
2020-04-13T12:31:12.25Z INFO vcdb.const DB user retrieved: vpxuser
2020-04-13T12:31:12.36Z INFO vcdb.const Retrieving DB password...
2020-04-13T12:31:12.112Z INFO vcdb.const DB password retrieved: ****
2020-04-13T12:31:12.121Z INFO vcdb.const Disabling jobs...

From the above Logs we can see that the upgrade process will be stuck at "Disabling Jobs." 

Process to isolate the Migration of vCenter being Stuck at "Exporting VCDB":

Running below SQL query in the VCDB using vpxuser credentials fails with error "SQLSERVERAGENT is not currently running so it cannot be notified of this action"
 

DECLARE @SQL AS VARCHAR(MAX);

 

SELECT @SQL = COALESCE(@SQL + CHAR(13), '') +

      'EXEC msdb..sp_update_job @job_name = '''+ CAST(name AS VARCHAR(100)) + ''', @enabled = 0;'

FROM msdb.dbo.sysjobs

where job_id in (select job_id

                 from MSDB.dbo.sysjobsteps

                 where database_name=db_name());

 

EXEC (@SQL);

Instructions to Run Above SQL Query:

1st Test:

1. Login to SQL management studio using the vpxuser account.
2. Navigate to Databases -> System Databases -> Select msdb -> Right Click -> New Query.
3. Execute above query.
4. Share the output.

2nd Test:

1. Login to SQL management studio using the SA account.
2. Navigate to Databases -> System Databases -> Select msdb -> Right Click -> New Query.
3. Execute above query.
4. Share the output.

Environment

VMware vCenter Server 6.7.x
VMware vCenter Server 6.0.x
VMware vCenter Server 6.5.x

Cause

The cause of the issue is that the "SQL Server Agent" service on the SQL DB machine was Stopped.

Resolution

Start the SQL Server Agent service on the SQL Server.