Guest Customization Specification passwords fail to decrypt after vCenter upgrade with error "ERR error:0407109F:rsa routines:RSA_padding_ check_PKCS1_type_2: pkcs decoding error"

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Guest Customization Specification passwords fail to decrypt after the vCenter upgrade.
• After upgrading to vCenter 6.0, users cannot customize guest virtual machines using an existing guest customization specification if the specification contains passwords.
• You receive a guest customization fault indicating that the password in the specification cannot be decrypted.
• In the /var/log/vmware/vpxd/vpxd.log file, you see entries similar to:
  
  

  YYYY-MM-DD HH:MM:SS error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] openssl error: error in doDecrypt()
  YYYY-MM-DD HH:MM:SS error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] ERR error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error
  YYYY-MM-DD HH:MM:SS error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] ERR error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed --> Context: decrypting password
  YYYY-MM-DD HH:MM:SS error vpxd[7FF8CE8E0700] [Originator@6876 sub=Default opID=5f6df3ff] [VmCustomizer] Error occured while creating deploy package. Msg: vim.fault.CannotDecryptPasswords
  YYYY-MM-DD HH:MM:SS info vpxd[7FF8CE8E0700] [Originator@6876 sub=Default opID=5f6df3ff] [VpxLRO] -- ERROR task-12424907 -- vm-706190 -- vim.VirtualMachine.customize: vim.fault.CannotDecryptPasswords: --> (vim.fault.CannotDecryptPasswords) 

Cause

This issue occurs due to the machine SSL key used to decrypt the passwords in the previous versions of vCenter Server.

Note: vCenter Server 6.0 release uses the VC solution key to decrypt the passwords.

Resolution

To resolve this issue, perform the steps below for each Guest Customization Specification after the vCenter upgrade:

1. Edit the Guest Customization Specification.
2. Re-enter any of the following that are included in the Guest Customization Specification:
   • Administrator Password.
   • Domain Join Password.
3. Save the updated Guest Customization Specification.
4. Guest Customization will be able to decrypt the passwords in the Guest Customization Specification.

For more information, see Edit a Customization Specification.

Additional Information

Impact/Risks:

• This applies only to any existing guest customization specifications that contain password.
• Other specifications are not affected.
• Other functionalities are not affected.