Establishing an outbound SSH session from Photon OS virtual machines to any target fails with the error "ssh_dispatch_run_fatal".

Products

• VMware Tanzu Kubernetes Grid

Issue/Introduction

Trying to open an SSH connection from the provider VM to a workload VM fails with the following error.

$ admin@test [ ~ ]$ ssh admin@172.##.0.##
ssh_dispatch_run_fatal: Connection to 172.##.0.# port 22: invalid argument

Ping communication is OK.
$ admin@test [ ~ ]$ ping 172.##.0.##
PING 172.##.0.## (172.##.0.##) 56(84) bytes of data.
64 bytes from 172.##.0.##: icmp_seq=1 ttl=49 time=3.05 ms
64 bytes from 172.##.0.##: icmp_seq=2 ttl=49 time=3.10 ms
64 bytes from 172.##.0.##: icmp_seq=3 ttl=49 time=3.04 ms
^C
--- 172.##.0.## ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms

Environment

Photon OS 4.x, 5.x

Cause

This occurs on OpenSSL FIPS  enabled  VM.

The default cipher list used by the SSH client is: "[email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]"

The first cipher in this list, [email protected], is not supported by OpenSSL in FIPS mode. Consequently, OpenSSL in FIPS mode raises an error immediately rather than allowing a fallback to the next cipher, which causes the connection error.

Resolution

 Add the cipher Ciphers aes128-ctr to the ssh config and restart the ssh daemon by following the below steps

1. Login as root to the VM
2. Edit the /etc/ssh/ssh_config file
3. Add the line 

   Host *
     Ciphers aes128-ctr

   
   [It forces the ssh to use Cipher aes128-ctr while establishing ssh]
4. systemctl restart sshd

Workaround:

Open SSH connection by specifying the cipher aes128-ctr

ssh username@IP -c aes128-ctr