When running a Policy Server and a Web Agent, what are the encryption algorithms used for communication between both?
The encryption for communication between the Web Agent and Policy Server is proprietary and depends on the FIPS mode in use (1).
For FIPS-Only, the algorithm is AES. For the other modes, AES and RC2. Neither SSL nor TLS is used. As noted above, the encryption is proprietary.
From Policy Server installation on Linux, the following information can be found. Here's an extract from the file available in the Policy Server installed files.
/{home_policy_server}/etpki-install/README_FIPS.txt :
CAPKI (ETPKI r5.x) and CA-OpenSSL support three modes of operation:
1) FIPS mode OFF 2) FIPS mode ON 3) FIPS Compatible mode
In FIPS mode OFF, all lower-level cryptographic operations are performed by the CA-OPENSSL default crypto implementation.
In FIPS mode ON, all lower-level cryptographic operations are delegated to OpenSSL FIPS Object Module.
FIPS Compatible mode is same as FIPS mode ON with the exception of password-based key derivation and encryption functions, that are allowed if FIPS approved algorithms are used.
This mode has been kept to facilitate smooth transitioning of product from NON-FIPS mode to pure FIPS mode.
When operating in FIPS mode ON or FIPS Compatible mode, only TLS is available. All versions of SSL are disabled due to usage of MD5 for key generation.
[...]
In CAPKI (ETPKI r5.x), flags for representing modes of operation are ETPKI_FIPS_MODE_OFF, ETPKI_FIPS_MODE_ON and ETPKI_FIPS_COMP_MODE respectively. [...]
These modes of operation are mutually exclusive. Hence, if a thread in an application initializes CAPKI (ETPKI r5.x) in FIPS mode ON, a subsequent thread in the same application whilst the first is still active will be unable to initialize in non-FIPS mode or FIPS Compatibility mode and must choose FIPS mode ON.
NB: This is a requirement of NIST for FIPS 140-2 certification and not a technical inadequacy
When operating in FIPS mode ON or FIPS Compatible mode, CAPKI (ETPKI r5.x) secure socket functionality defaults to TLS, whereas in FIPS mode OFF, it defaults to SSL_V23.
[...]
The function etpki_cert_get_fingerprint() will always return an error in FIPS mode ON or FIPS Compatible mode as the MD5 digest is not FIPS approved. The SHA-1 digest utilizing equivalent, etpki_cert_get_thumbprint() however will operate in both FIPS and non-FIPS modes.
In CA-OpenSSL, flags for representing modes of operation are OPENSSL_FIPS_MODE_OFF, OPENSSL_FIPS_MODE_ON and OPENSSL_FIPS_COMP_MODE respectively. [...]
When operating in FIPS mode, the SSL library only supports TLS. All SSL modes fail due to non-FIPS compliance.
The following non-FIPS approved ciphers will fail to initialize when operating in FIPS mode ON or FIPS Compatible mode:
DESX RC4 RC2 IDEA BLOWFISH CAST5 RC5 The following non-FIPS approved digests will fail to initialize when operating in FIPS mode ON or FIPS compatible mode:
MD5 MD4 MD2 SHA-0 RIPEMD160
Cryptographic operations which involve the use of any of the previous disabled security functions will also fail, i.e. NID_pbeWithMD5AndDES_CBC, NID_md5WithRSAEncryption, etc.