SSH is not a supported protocol for direct serial port backing in VMware vSphere ESXi.
VMware vSphere ESX 8.x
When configuring a virtual machine serial port for network connections in VMware vSphere ESXi, SSH is not a supported protocol for direct serial port backing.
While Telnet is commonly used telnet://<SERVER>:23 there are more secure alternatives include Telnet over SSL and URI authentication parameters. See Authentication Parameters for Virtual Serial Port Network Connections for more details.