Seeing weird errors in salt master for StreamClosed exception

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

There are error messages appearing in the salt-master and salt-api logs:

xxx [salt.master      :xxx][ERROR   ][xxx] Error in function _return:
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/master.py", line 1927, in run_func
    ret = getattr(self, func)(load)

...

    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
salt.ext.tornado.iostream.StreamClosedError: Stream is closed

Environment

salt-master and salt-api 3006.10

Resolution

• The workaround is to restart the salt-master and wait 10 minutes before submitting new jobs
• There is also a fix targeted at the salt-master in 3006.15