VCSA Upgrade to 7.0 fails at wcp-firstboot
search cancel

VCSA Upgrade to 7.0 fails at wcp-firstboot

book

Article ID: 316583

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
  • Upgrading vCenter Server to 7.0 fails with WCP firstboot with error "Failed to create service account for workload storage" "VMware directory error[9127]"
  • You will see the log entries similar to the below snippet in /var/log/firstboot

wcp-firstboot.py_xxxxxx_stderr.log
2021-02-02T13:21.231Z WCP firstboot failed

wcp-firstboot.py_xxxx_stdout.log
2020-12-02T13:21:35.279Z INFO wcp-firstboot WCP storage user does not exists, create the user.
2020-12-02T13:21:35.279Z INFO wcp-firstboot Creating ServiceAccount client...
2020-12-02T13:21:35.368Z Further filtering retrieved service registration list on hostname : vcenter.domain.local
2020-12-02T13:21:35.376Z INFO wcp-firstboot Creating service account...
2020-12-02T13:21:35.377Z INFO wcp-firstboot Initializing ServiceAccount session...
2020-12-02T13:21:43.727Z ERROR wcp-firstboot Unexpected error creating ServiceAccount {messages : [LocalizableMessage(id='com.vmware.vcenter.svcaccountmgmt.error', default_message='Exception found (Internal Server Error, VMware directory error[9127])', args=['Internal Server Error, VMware directory error[9127]'], params=None, localized=None)], data : None, error_type : None}
2020-12-02T13:21:43.727Z ERROR wcp-firstboot Failed to create service account for workload storage
Traceback (most recent call last):
  File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 298, in _create_storage_user
    password = svcacctmgmt_client.create_svc_account(self._user_name)
  File "/usr/lib/vmware-wcp/py-modules/svcacctmgmt.py", line 90, in create_svc_account
    raise er
  File "/usr/lib/vmware-wcp/py-modules/svcacctmgmt.py", line 84, in create_svc_account
    svcacct_pwd_out = svcacct_client.create(create_spec)
  File "/usr/lib/vmware-wcp/py-modules/vapi-bindings/com/vmware/vcenter/svcaccountmgmt_client.py", line 368, in create
    'create_spec': create_spec,
  File "/usr/lib/vmware-vapi/lib/python/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py", line 345, in _invoke
    return self._api_interface.native_invoke(ctx, _method_name, kwargs)
  File "/usr/lib/vmware-vapi/lib/python/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py", line 298, in native_invoke
    self._rest_converter_mode)
com.vmware.vapi.std.errors_client.Error: {messages : [LocalizableMessage(id='com.vmware.vcenter.svcaccountmgmt.error', default_message='Exception found (Internal Server Error, VMware directory error[9127])', args=['Internal Server Error, VMware directory error[9127]'], params=None, localized=None)], data : None, error_type : None}

 


Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server 7.0.x

Cause

This issue is caused due to the mismatch between DC account and the SAM account in the vCenter Server.

Resolution

VMware Engineering Team is aware of this issue and working towards the fix.



    Workaround:
    To resolve this issue follow the below steps to identify if the DC account and SAM account names are different.
    1. Check the SAM Account for the VC using below command,
    ldapsearch -H ldap://localhost -x -W -D "cn=administrator,cn=users,dc=vsphere,dc=local" -b "ou=Domain Controllers,dc=vsphere,dc=local"
    # Domain Controllers, vsphere.local
    dn: ou=Domain Controllers,dc=vsphere,dc=local

    objectClass: top
    objectClass: organizationalUnit
    ou: Domain Controllers
    nTSecurityDescriptor:: AQAEgBQAAAA0AAAAAAAAAEQAAAABBgAAAAAABxUAAAAAAAAAiIgAAAA
     AAAIAAAAA9AEAAAECAAAAAAAFIAAAACACAAACAGAAAGAAwAAAAAQIAAAAAAAcgAAAAmgIA
     AAAAGAAzAAAgAQIAAAAAAAUgAAAAIAIAAAAAKAAzAAAgAQYAAAAAAAcVAAAAAAAAQBAAA=

    # vcenter.domain.local, Domain Controllers, vsphere.local
    dn: cn=vcenter.domain.local,ou=Domain Controllers,dc=vsphere,dc=local
    vmwPlatformServicesControllerVersion: 6.7.0
    krbPrincipalKey:: MIGboAMCAQGhAwIBAKIDAgEBpIGJMIGGMEmhRzBFoAM7WFFJ2YM
     DkPsyujMngIrLNXfAnPEBR4flMFfQ29beixwNwG/4sOwIj+bER1lVMibq5x+A7h5x95MDmhN
     zA1oAMCARehLgQsO5GdCElPg/GNGhC8gQaAmN6cBLyhX64xkpMiOFjspXwoE+11TEY=
    userPrincipalName: vcenter2.domain.local@VSPHERE.LOCAL
    cn: vcenter2.domain.local
    objectClass: top
    objectClass: person
    objectClass: organizationalPerson
    objectClass: user
    objectClass: computer
    nTSecurityDescriptor:: AQAEgBQAAAA0AAAAAAAAAEQAAAABBgAAAAAABxUAAAAAAAAAiIgAAAA
     AAAIAAAAA9AEAAAECAAAAAAAFIAAAACACAAACAGAAAGAAwAAAAAQIAAAAAAAcgAAAAmgIA
     AAAAGAAzAAAgAQIAAAAAAAUgAAAAIAIAAAAAKAAzAAAgAQYAAAAAAAcVAAAAAAAAQBAAA=
    vmwMachineGUID: c5efd22d-5f31-44ea-94c2-1599f48321ec
    userAccountControl: 0
    sAMAccountName: vcenter.domain.local
    1. Check the pnid as well using below command
      • /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
      • vcenter.domain.local
    2. In the above scenario the DC account is vcenter2.domain.local and SAM account is vcenter.domain.local
    3. Initiate the VC Upgrade process, post stage 1 take a snapshot of the newly deployed VC.
    4. Enable putty/ssh session on the new vCenter server.
    5. Open putty session on new VC and add an entry in the /etc/hosts
      • Add  <Source VC IP Address> <CN Name>
      • Example:192.168.10.13 vcenter2.domain.local
    6. Continue the upgrade from the Stage 2 or re-initiate the upgrade from VAMI page and use the temporary IP address. You should be able to complete the VC upgrade successfully.


    Powered by Wolken Software