Unavailable Hardware Security Module (HSM) Can Prevent the CA API Gateway from Starting

Products

CA API Gateway

Issue/Introduction

The CA API Gateway will not return to a functional state, after a server restart, if the HSM module is unavailable. This can be verified by the API Gateway's inability to process traffic and/or the following errors present in the SSG log:

^{2017-08-22T13:29:01.564-0500 WARNING 1 STDERR: at com.l7tech.util.DefaultMasterPasswordFinder.findMasterPasswordBytes(DefaultMasterPasswordFinder.java:42)}
^{2017-08-22T13:29:01.570-0500 WARNING 1 STDERR: at com.l7tech.util.L7C2SecretEncryptor.decryptPassword(L7C2SecretEncryptor.java:136)}
^{2017-08-22T13:29:01.575-0500 WARNING 1 STDERR: at com.l7tech.util.MasterPasswordManager.decryptPasswordIfEncrypted(MasterPasswordManager.java:225)}
^{2017-08-22T13:29:01.581-0500 WARNING 1 STDERR: at com.l7tech.server.util.PropertiesDecryptor.decryptEncryptedPasswords(PropertiesDecryptor.java:49)}
^{2017-08-22T13:29:01.586-0500 WARNING 1 STDERR: at com.l7tech.server.util.PasswordDecryptingPropertiesFactoryBean.mergeProperties(PasswordDecryptingPropertiesFactoryBean.java:44)}
^{2017-08-22T13:29:01.592-0500 WARNING 1 STDERR: at org.springframework.beans.factory.config.PropertiesFactoryBean.createInstance(PropertiesFactoryBean.java:113)}
^{2017-08-22T13:29:01.597-0500 WARNING 1 STDERR: at org.springframework.beans.factory.config.PropertiesFactoryBean.createProperties(PropertiesFactoryBean.java:98)}
^{2017-08-22T13:29:01.603-0500 WARNING 1 STDERR: at org.springframework.beans.factory.config.PropertiesFactoryBean.afterPropertiesSet(PropertiesFactoryBean.java:69)}
^{2017-08-22T13:29:01.608-0500 WARNING 1 STDERR: at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$5.run(AbstractAutowireCapableBeanFactory.java:1469)}
^{2017-08-22T13:29:01.614-0500 WARNING 1 STDERR: at java.security.AccessController.doPrivileged(Native Method)}
^{2017-08-22T13:29:01.619-0500 WARNING 1 STDERR: at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1467)}
^{2017-08-22T13:29:01.625-0500 WARNING 1 STDERR: at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1419)}
^{2017-08-22T13:29:01.631-0500 WARNING 1 STDERR: ... 31 more}
^{2017-08-22T13:29:01.636-0500 WARNING 1 STDERR: Caused by: com.ncipher.provider.nCRuntimeException: com.ncipher.km.nfkm.nfkmCommunicationException error (st=ServerNotRunning) : NFKM_getinfo}
^{2017-08-22T13:29:01.642-0500 WARNING 1 STDERR: at com.ncipher.provider.km.nCipherKM.getSW(nCipherKM.java:597)}
^{2017-08-22T13:29:01.647-0500 WARNING 1 STDERR: at com.ncipher.provider.km.KMKeyStore.engineLoad(KMKeyStore.java:818)}
^{2017-08-22T13:29:01.653-0500 WARNING 1 STDERR: at java.security.KeyStore.load(KeyStore.java:1214)}
^{2017-08-22T13:29:01.658-0500 WARNING 1 STDERR: at com.l7tech.util.KeyStorePrivateKeyMasterPasswordFinder.createDecryptionBag(KeyStorePrivateKeyMasterPasswordFinder.java:181)}
^{2017-08-22T13:29:01.664-0500 WARNING 1 STDERR: at com.l7tech.util.KeyStorePrivateKeyMasterPasswordFinder.findMasterPasswordBytes(KeyStorePrivateKeyMasterPasswordFinder.java:158)}
^{2017-08-22T13:29:01.669-0500 WARNING 1 STDERR: at com.l7tech.util.DefaultMasterPasswordFinder.findMasterPasswordBytes(DefaultMasterPasswordFinder.java:38)}
^{2017-08-22T13:29:01.675-0500 WARNING 1 STDERR: ... 42 more}
^{2017-08-22T13:29:01.680-0500 WARNING 1 STDERR: Caused by: com.ncipher.km.nfkm.nfkmCommunicationException: error (st=ServerNotRunning) : NFKM_getinfo}
^{2017-08-22T13:29:01.688-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.Command.stop(Command.java:338)}
^{2017-08-22T13:29:01.693-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.Command.waitReply(Command.java:506)}
^{2017-08-22T13:29:01.699-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.Command._go(Command.java:260)}
^{2017-08-22T13:29:01.705-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.Command._go(Command.java:268)}
^{2017-08-22T13:29:01.710-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.GetInfo.go(GetInfo.java:50)}
^{2017-08-22T13:29:01.716-0500 WARNING 1 STDERR: at com.ncipher.km.nfkm.GetInfo.saveExistingObjects(GetInfo.java:354)}

^{**** Unable to start the server: Error starting server : Error creating bean with name 'org.springframework.beans.factory.config.PropertyOverrideConfigurer#0' defined in class path resource [com/l7tech/server/resources/dataAccessContext.xml]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanExpressionException: Expression parsing failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'hibernateProperties' defined in class path resource [com/l7tech/server/resources/dataAccessContext.xml]: Invocation of init method failed; nested exception is java.lang.RuntimeException: Unable to instantiate master password finder with File arg: com.ncipher.km.nfkm.nfkmCommunicationException error (st=ServerNotRunning) : NFKM_getinfo}

Environment

This impacts any API Gateway appliances with a Hardware Security Module (HSM) attached.

Resolution

If a Hardware Security Module (HSM) is installed on the Gateway, verify the module is physically secure, running, and configured per the product documentation.

Additional Information

API Gateway documentation on configuration of the HSM: https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-gateway/10-1/install-configure-upgrade/configure-the-appliance-gateway/configure-hardware-security-modules-hsm.html