How to enable FIPS (Federal Information Processing Standards) on vSphere Replication appliance

Products

VMware Live Recovery

Issue/Introduction

vSphere Replication 8.5 has introduced FIPS support.

This article provides steps to enable vSphere Replication to run in FIPS compliant mode.

Environment

VMware vSphere Replication 8.5.x

Resolution

Steps to enable FIPS mode manually in the vSphere Replication appliance

It is recommended to connect to the vSphere Replication appliance via ssh (see KB2112307), however this changes can be performed via the console.
We assume familiarity with vi and the ability to edit file (see KB1020302 for additional details)
Edit /opt/vmware/hms/bin/java-fips-options.sh and uncomment JAVA_FIPS_OPTIONS

Edit /etc/vmware/hbrsrv.xml
Add a tag to enable fips:

<Config>
    <vmacore>
        <ssl>
            <fips>true</fips>
        </ssl>
    </vmacore>
</Config>

Edit /opt/vmware/dr/conf/drconfig.xml
Add a tag to enable fips:

<Config>
    <vmacore>
        <ssl>
            <fips>true</fips>
        </ssl>
    </vmacore>
</Config>

Restart the his and her services:

systemctl restart hms
systemctl restart hbrsrv

As the root user (details)
Edit /boot/grub/grub.cfg
in each "menuentry" section find the line that starts with "linux" and add "fips=1"
i.e:

... Example ...
menuentry "Photon" {
    linux /$photon_linux root=$rootpartition $photon_cmdline coredump_filter=0x37 $systemd_cmdline fips=1
    if [ -f /$photon_initrd ]; then
        initrd /$photon_initrd
    fi
}
...

Enabling strict mode for the Replication UI:
1. Edit /usr/lib/systemd/system/dr-client.service - comment out the existing Environment='CATALINA_OPTS=-Xms768m -Xmx1024m' and uncomment the one under # Uncomment to enable FIPS. The file fragment should look like this:
```
#Environment='CATALINA_OPTS=-Xms768m -Xmx1024m'
# Uncomment to enable FIPS
Environment='CATALINA_OPTS=-Xms768m -Xmx1024m -Djava.security.properties==/opt/vmware/dr-client/conf/vmware-override-java.security -Djava.ext.dirs=/opt/vmware/dr-client/lib/ext -Dorg.bouncycastle.jca.enable_jks=true -Dorg.bouncycastle.fips.approved_only=true'
```
2. Uncomment the <Manager> tag in /opt/vmware/dr-client/conf/context.xml - the file fragment with the tag should look like this:
```

<Manager pathname="" secureRandomAlgorithm=""/>
```
3. Edit /opt/vmware/dr-client/lib/h5dr.properties and modify keyStoreName to point to BCFKS format keystore which contains root CA certificates. The property should look like this:
```
keyStoreName=hms-keystore.bks
```
  If you choose to use a truststore other than the default one, link to it should be added to /opt/vmware/dr-client/lib/ or /opt/vmware/dr-client/webapps/dr/WEB-INF/classes/. Keystore format should be BCFKS. To import it from JKS format use the following command:
```
$JAVA_HOME/bin/keytool -importkeystore -srckeystore <path-to-jks-keystore> -srcstoretype JKS -srcstorepass <keystorepass> -destkeystore <path-to-target-bks-keystore> -deststoretype BCFKS -deststorepass <keystorepass> -provider org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath /opt/vmware/dr-client/lib/ext/bc-fips-1.0.2.jar
```
  NOTE: The path to the keystore file should be accessible and the keystore should have Read permission for 'tomcat' user
```
setfacl -m "u:tomcat:x" /path_to_keystore/
setfacl -m "u:tomcat:r" /path_to_keystore/keystore_file
```
  Restart dr-client (only if FIPS has already been enabled for the appliance, otherwise skip this step):
```
systemctl daemon-reload; systemctl restart dr-client
```

Start Config UI in strict mode:

Edit /usr/lib/systemd/system/drconfigui.service - comment out the existing Environment=...' and uncomment the one under # Uncomment to enable FIPS. The file fragment should look like this:

#Environment='CATALINA_OPTS=-Xms128m -Xmx256m -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCCause -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2m -Xloggc:/opt/vmware/drconfigui/logs/drconfigui-gc.log'
# Uncomment to enable FIPS
Environment='CATALINA_OPTS=-Xms128m -Xmx256m -Djava.security.properties==/opt/vmware/drconfigui/conf/vmware-override-java.security -Djava.ext.dirs=/opt/vmware/drconfigui/lib/ext -Dorg.bouncycastle.jca.enable_jks=true -Dorg.bouncycastle.fips.approved_only=true -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCCause -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2m -Xloggc:/opt/vmware/drconfigui/logs/drconfigui-gc.log'

Uncomment the <Manager> tag in /opt/vmware/drconfigui/conf/context.xml - the file fragment with the tag should look like this:
```

<Manager pathname="" secureRandomAlgorithm=""/>
```
Restart drconfigui (only if FIPS has already been enabled for the appliance, otherwise skip this step):
```
systemctl daemon-reload; systemctl restart drconfigui
```

Reboot the appliance
Note: SSHD will read that the kernel has enabled FIPS mode and will activate it too. No need to edit anything in the sshd configuration.

Validate FIPS mode is enabled

Validate the kernel commandline.
```
cat /proc/cmdline
```
Validate that kernel has enabled FIPS mode.
```
cat /proc/sys/crypto/fips_enabled
```
Validate that SSHD has enabled FIPS mode.
```
journalctl -u sshd -b 0 | grep "FIPS"
```
Validate that dr-configurator has enabled FIPS mode.
```
grep "FIPS" /var/log/vmware/dr/drconfig*
```
Validate that VRS is running in FIPS mode.
```
grep FIPS /var/log/vmware/hbrsrv.log
```
Note: The output should contain a line with the following text '...Service is running in FIPS mode.'
Validate UI strict mode.
All UI features should be available and work as expected