When running a Policy Server and a Web Agent, what are the encryption algorithms used for communication between both?
The encryption for communication between the Web Agent and Policy Server is proprietary and depends on the FIPS mode in use (1).
For FIPS-Only, the algorithm is AES. For the other modes, AES and RC2. Neither SSL nor TLS is used. As noted above, the encryption is proprietary.
From Policy Server installation on Linux, the following information can be found. Here's an extract from the file available in the Policy Server installed files.
/{home_policy_server}/etpki-install/README_FIPS.txt :
CAPKI (ETPKI r5.x) and CA-OpenSSL support three modes of operation:
1) FIPS mode OFF
2) FIPS mode ON
3) FIPS Compatible mode
In FIPS mode OFF, all lower-level cryptographic operations are
performed by the CA-OPENSSL default crypto implementation.
In FIPS mode ON, all lower-level cryptographic operations are
delegated to OpenSSL FIPS Object Module.
FIPS Compatible mode is same as FIPS mode ON with the exception of
password-based key derivation and encryption functions, that are
allowed if FIPS approved algorithms are used.
This mode has been kept to facilitate smooth transitioning of
product from NON-FIPS mode to pure FIPS mode.
When operating in FIPS mode ON or FIPS Compatible mode, only TLS is
available. All versions of SSL are disabled due to usage of MD5 for
key generation.
[...]
In CAPKI (ETPKI r5.x), flags for representing modes of operation are
ETPKI_FIPS_MODE_OFF, ETPKI_FIPS_MODE_ON and ETPKI_FIPS_COMP_MODE
respectively.
[...]
These modes of operation are mutually exclusive. Hence, if a thread
in an application initializes CAPKI (ETPKI r5.x) in FIPS mode ON, a
subsequent thread in the same application whilst the first is still
active will be unable to initialize in non-FIPS mode or FIPS
Compatibility mode and must choose FIPS mode ON.
NB: This is a requirement of NIST for FIPS 140-2 certification and
not a technical inadequacy
When operating in FIPS mode ON or FIPS Compatible mode, CAPKI (ETPKI
r5.x) secure socket functionality defaults to TLS, whereas in FIPS
mode OFF, it defaults to SSL_V23.
[...]
The function etpki_cert_get_fingerprint() will always return an
error in FIPS mode ON or FIPS Compatible mode as the MD5 digest is
not FIPS approved. The SHA-1 digest utilizing equivalent,
etpki_cert_get_thumbprint() however will operate in both FIPS and
non-FIPS modes.
In CA-OpenSSL, flags for representing modes of operation are
OPENSSL_FIPS_MODE_OFF, OPENSSL_FIPS_MODE_ON and
OPENSSL_FIPS_COMP_MODE respectively.
[...]
When operating in FIPS mode, the SSL library only supports TLS. All
SSL modes fail due to non-FIPS compliance.
The following non-FIPS approved ciphers will fail to initialize when
operating in FIPS mode ON or FIPS Compatible mode:
DESX
RC4
RC2
IDEA
BLOWFISH
CAST5
RC5
The following non-FIPS approved digests will fail to initialize when
operating in FIPS mode ON or FIPS compatible mode:
MD5
MD4
MD2
SHA-0
RIPEMD160
Cryptographic operations which involve the use of any of the
previous disabled security functions will also fail,
i.e. NID_pbeWithMD5AndDES_CBC, NID_md5WithRSAEncryption, etc.