Data Source Addition Fails with "Unwrapping failed" on Collector in VCF Operations for Networks

Products

• VCF Operations for Networks

Issue/Introduction

• No flows seen in VCF Operations for Networks GUI
  
  
• Unable to add VC and NSX data source to VCF Operations for Networks via  VCF Operations GUI.

• Data source addition fails on a specific collector (other collectors in the same deployment may work fine).

• The following error appears in the collector logs:

  YYYY-MM-1DD19:12:43.000020Z ERROR collector 42124 [netw@4413 class="common.utils.CommonUtils" thread="collector-process-msg-exec-5" method="logException" line="2728"] Processing message from saas threw exception. RequestId=########-######-####-#####-########, Type=VALIDATE_DP_CREDENTIAL, TenantId=17648
  java.security.InvalidKeyException: Unwrapping failed
          at com.sun.crypto.provider.RSACipher.engineUnwrap(RSACipher.java:529) _[_:_]
          at javax.crypto.Cipher.unwrap(Cipher.java:2590) _[_:_]
          at com.vnera.crypto.AsymmetricCryptWithSymmetricKey_Decrypt.decrypt(AsymmetricCryptWithSymmetricKey.java:138) _[common-0.001-SNAPSHOT.jar:_]
          at com.vnera.crypto.AsymmetricCrypt_Decrypt.decrypt(AsymmetricCrypt.java:90) _[common-0.001-SNAPSHOT.jar:_]
          at com.vnera.crypto.AsymmetricCrypt_Decrypt.decodeAndDecrypt(AsymmetricCrypt.java:97) _[common-0.001-SNAPSHOT.jar:_]
          at com.vnera.collector.config.protobufs.DataProviderConfigurationHelper.decryptPasswordsInConnData(DataProviderConfigurationHelper.java:69) _[contracts-0.001-SNAPSHOT.jar:_]
          at com.vnera.collector.core.engine.SaasCommandProcessor.decryptConnData(SaasCommandProcessor.java:1344) _[collector-0.001-SNAPSHOT.jar:_]
          at com.vnera.collector.core.engine.SaasCommandProcessor.processMessage(SaasCommandProcessor.java:379) _[collector-0.001-SNAPSHOT.jar:_]
          at com.vnera.collector.core.saascommunication.SaasListener.lambda_receiveMessage_0(SaasListener.java:116) _[collector-0.001-SNAPSHOT.jar:_]
          at java.util.concurrent.FutureTask.run(FutureTask.java:264) [_:_]
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [_:_]
          at java.util.concurrent.ThreadPoolExecutor_Worker.run(ThreadPoolExecutor.java:635) [_:_]
          at java.lang.Thread.run(Thread.java:840) [_:_]
  Caused by: javax.crypto.BadPaddingException: Padding error in decryption
          at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:389) _[_:_]
          at com.sun.crypto.provider.RSACipher.engineUnwrap(RSACipher.java:519) _[_:_]
          ... 12 more

• Log location: /var/log/arkin/collector/collector.STDOUT-*.log

Environment

VCF Operations for Networks 9.0.x

Cause

• Data source addition or credential validation fails on a specific collector due to a mismatch between the collector's internal encryption key and the certificate stored on the platform.

• The platform uses this certificate to encrypt credentials before sending them to the collector. If they don't match, the collector cannot decrypt the credentials.

Resolution

If you have encountered this issue then Open a support case with Broadcom Support using the directions at KB 142884 - Creating and managing Broadcom cases to review your VCF Operations for Networks deployment.