gpcrondump Failed with Error "'ssh_exchange_identification: read: Connection reset by peer"

Products

• VMware Tanzu Greenplum

Issue/Introduction

Symptoms:

gpcrondump failed with the following error message:

20160701:14:34:01:055166 gpcrondump:linux:dbuser-[ERROR]:-ExecutionError: 'non-zero rc: 255' occured. Details: 'ssh -o 'StrictHostKeyChecking no' localhost ". /opt/ossdata/greenplum-db/./greenplumpath.sh; $GPHOME/sbin/gpoperation.py"'
cmd had rc=255 completed=True halted=False
stdout=''
stderr='ssh_exchange_identification: read: Connection reset by peer
'
Traceback (most recent call last):
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 211, in run
self.cmd.run()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/operations/init_.py", line 49, in run
self.ret = self.execute()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/operations/utils.py", line 48, in execute
cmd.run(validateAfter=True)
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 702, in run
self.validate()
File "/opt/ossdata/greenplum-db/lib/python/gppylib/commands/base.py", line 737, in validate
raise ExecutionError("non-zero rc: %d" % self.results.rc, self)
ExecutionError: ExecutionError: 'non-zero rc: 255' occured. Details: 'ssh -o 'StrictHostKeyChecking no' localhost ". /opt/ossdata/greenplum-db/./greenplum_path.sh; $GPHOME/sbin/gpoperation.py"' cmd had rc=255 completed=True halted=Fals
e
stdout=''
stderr='ssh_exchange_identification: read: Connection reset by peer

Environment

Cause

This issue happened on a single node cluster, with six primary instances. The backup always fails when running in the afternoon (while there are user connections). This SSH error could be caused by many reasons, one of them being too many SSH connections. 

 

Resolution

Set the parameter Maxstartups to 100 in the /etc/ssh/sshd_config file.