Modifying vDS membership fails after upgrading from vCenter Server 5.1 to 5.5 or 5.5 to 6.x
search cancel

Modifying vDS membership fails after upgrading from vCenter Server 5.1 to 5.5 or 5.5 to 6.x

book

Article ID: 331378

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:

After upgrading from vCenter Server 5.1 to 5.5 or vCenter Server 5.1/5.5 to 6.x and importing the vSphere distributed switch, you see these symptoms:

  • Adding ESXi hosts or virtual machines to vSphere Distributed Switch fails.
  • In the C:\Documents and Settings\All Users\Application Data\VMware\VirtualCenter\logs\vpxd.log file, you see entries similar to:

    T10:03:13.714+02:00 [12784 warning 'Default' opID=7E4DC15E-0000D3FA-bc] [VdbStatement] SQL execution failed: INSERT INTO VPX_DVPORT_MEMBERSHIP WITH (ROWLOCK) (DVS_ID, DVPORT_KEY, >DVPORTGROUP_ID, HOST_ID, LAG_KEY) VALUES (?, ?, ?, ?, ?)
    2014-07-08T10:03:13.714+02:00 [12784 warning 'Default' opID=7E4DC15E-0000D3FA-bc] [VdbStatement] Execution elapsed time: 1 ms
    ...
    T10:03:13.716+02:00 [12784 error 'Default' opID=7E4DC15E-0000D3FA-bc] [VdbStatement] SQLError was thrown: "ODBC error: (23000) - [Microsoft][SQL Server Native Client 11.0][SQL Server]Violation of >PRIMARY KEY constraint 'PK_VPX_DVPORT_MEMBERSHIP'. Cannot insert duplicate key in object 'dbo.VPX_DVPORT_MEMBERSHIP'. The duplicate key value is (916, 2)." is returned when executing SQL statement "INSERT INTO
  • Attempts to make changes to the membership of the affected vSphere Distributed Switch fails and restart vCenter Server.
  • In the C:\Documents and Settings\All Users\Application Data\VMware\VirtualCenter\logs\vpxd.log file, you see entries similar to:

    T10:03:13.828+02:00 [12784 error 'commonvpxCommon' opID=7E4DC15E-0000D3FA-bc] [Vpxd_HandleVmRootError] Received unrecoverable VmRootError. Generating minidump ...
    2014-07-08T10:03:13.828+02:00 [12784 error 'Default' opID=7E4DC15E-0000D3FA-bc] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23000) - [Microsoft][SQL Server Native Client 11.0][SQL Server]Violation of PRIMARY KEY constraint 'PK_VPX_DVPORT_MEMBERSHIP'. Cannot insert duplicate key in object 'dbo.VPX_DVPORT_MEMBERSHIP'. The duplicate key value is (916, 2)." is returned when executing SQL statement "INSERT INTO VPX_DVPORT_MEMBERSHIP WITH (ROWLOCK) (DVS_ID, DVPORT_KEY, DVPORTGROUP_ID, HOST_ID, LAG_KEY) VALUES (?, ?, ?, ?, ?)"
     

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.


Environment

VMware vCenter Server Appliance 5.5.x
VMware vCenter Server 6.5.x
VMware vCenter Server 5.5.x
VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.0.x
VMware vCenter Server 5.x - View
VMware vCenter Server 6.0.x

Cause

This issue occurs due to the value of PORT_COUNTER in the VPX_DVS table having changed to a number less than the maximum key value already used. This causes a problem when it tries to add a new port, it uses a key value that already exists and fails.

Resolution

This issue is resolved in vCenter Server 6.0 Update 3 and vCenter Server 6.5 Update 1
 


    Workaround:
    To work around this issue, re-deploy the affected vSphere Distributed Switch. Alternatively, this issue can be fixed directly in the vCenter Server database.
     
    To work around this issue directly in vCenter Server database:

    Note: Ensure to stop vCenter Server and back up the database before proceeding with these steps.
    1. Identify the affected vSphere Distributed Switch by looking for this error message in the vCenter Server vpxd log file around the time of the failure. You must see an error that includes a string similar to:

      Cannot insert duplicate key in object 'dbo.VPX_DVPORT_MEMBERSHIP'. The duplicate key value is (916,2)

      Note: In this case, the important parts are the table prefix and the first number in parenthesis, as this tells us in this example that 916 is the affected switch's ID and dbo is the table prefix. Make a note of these values.

      You can also use windows event viewer vpxd crash events to find the duplicate key error and DVS_ID Value.
       
    2. Log in to the SQL Management studio with an administrative user.
    3. Expand the databases on the left and double-click the vCenter Server database (default name VCDB).
    4. Run this SQL statement:

      For Microsoft SQL:

      UPDATE VPX_DVS SET PORT_COUNTER=((SELECT MAX( DVPORT_KEY+0 ) FROM VPX_DVPORT_MEMBERSHIP WHERE DVS_ID='DVS_ID' )+1) WHERE ID='DVS_ID';

      For vPostgres SQL (vCenter Server Appliance):

      UPDATE VPX_DVS SET PORT_COUNTER=((SELECT MAX(CAST(DVPORT_KEY AS INT)) FROM VPX_DVPORT_MEMBERSHIP WHERE DVS_ID='DVS_ID')+1) WHERE ID='DVS_ID';

      Note: The DVS_ID needs to be changed to match the DVS_ID reported in the output from the error message.
       
    5. Repeat all steps for each affected distributed switch.


    Additional Information

    To be alerted when this document is updated, click the Subscribe to Article link in the Actions box vCenter Server 5.1 から 5.5、または 5.5 から 6.0 へのアップグレード後に vDS メンバーシップを変更すると失敗する
    从 vCenter Server 5.1 升级到 5.5 或从 5.5 升级到 6.0 后，修改 vDS 成员资格失败

    Powered by Wolken Software