"ERR error:0407109F:rsa routines:RSA_padding_ check_PKCS1_type_2: pkcs decoding error" Guest Customization Specification passwords fail to decrypt after vCenter upgrade

Products

• VMware

Issue/Introduction

Symptoms:

• Guest Customization Specification passwords fail to decrypt after vCenter upgrade.
• After upgrading to vCenter 6.0, user cannot customize guest virtual machines using an existing guest customization specifications, if the specification contains passwords.
• You receive a guest customization fault indicating that the password in the specification cannot be decrypted.
• In the /var/log/vmware/vpxd/vpxd.log file, you see entries similar to:

2017-09-14T18:09:10.502Z error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] openssl error: error in doDecrypt()
2017-09-14T18:09:10.504Z error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] ERR error:0407109F:rsa routines:RSA_padding_check_PKCS1_type_2:pkcs decoding error
2017-09-14T18:09:10.504Z error vpxd[7FF8CE8E0700] [Originator@6876 sub=vpxCrypt opID=5f6df3ff] [void VpxKey::Decrypt(const std::vector<unsigned char>&, std::vector<unsigned char>&, int, bool)] ERR error:04065072:rsa routines:RSA_EAY_PRIVATE_DECRYPT:padding check failed --> Context: decrypting password
2017-09-14T18:09:10.511Z error vpxd[7FF8CE8E0700] [Originator@6876 sub=Default opID=5f6df3ff] [VmCustomizer] Error occured while creating deploy package. Msg: vim.fault.CannotDecryptPasswords
2017-09-14T18:09:10.516Z info vpxd[7FF8CE8E0700] [Originator@6876 sub=Default opID=5f6df3ff] [VpxLRO] -- ERROR task-12424907 -- vm-706190 -- vim.VirtualMachine.customize: vim.fault.CannotDecryptPasswords: --> (vim.fault.CannotDecryptPasswords) {
 

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Cause

This issue occurs due to the machine SSL key used to decrypt the passwords in the previous versions of vCenter Server.

Note: vCenter Server 6.0 release uses the VC solution key to decrypt the passwords.

Resolution

To resolve this issue, perform the below steps for each Guest Customization Specification after the vCenter upgrade:

1. Edit the Guest Customization Specification.
2. Re-enter any of the following that are included in the Guest Customization Specification:
   • Administrator Password.
   • Domain Join Password.
3. Save the updated Guest Customization Specification.
4. Guest Customization will be able to decrypt the passwords in the Guest Customization Specification.

For more information, see Edit a Customization Specification.

Additional Information

Impact/Risks:

• This applies only to any existing guest customization specifications that contain password.
• Other specifications are not affected.
• Other functionalities are not affected.