Unable to receive any emails from the vCenter

Products

• VMware vCenter Server

Issue/Introduction

No recent changes in the environment and email alerts from the vCenter stopped all of a sudden

Environment

/var/log/VMware/messages

2024-06-12T00:17:50.417062-07:00 vCenter sendmail[73281]: 45C7HoRN073281: 45C7HoRO073281: DSN: [email protected];[email protected]... Invalid route address
2024-06-12T00:17:50.418801-07:00 vCenter sendmail[73281]: 45C7HoRO073281: [email protected], delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=32145, relay=[127.0.0.1] [127.0.0.1], dsn=4.4.1, stat=Deferred: Connection refused by [127.0.0.1]
2024-06-12T00:17:53.413693-07:00 vCenter sendmail[73892]: 45C7HrFM073892: [email protected], size=1121, class=0, nrcpts=1, msgid=<[email protected]>, relay=root@localhost
2024-06-12T00:17:53.413782-07:00 vCenter sendmail[73892]: 45C7HrFM073892: 45C7HrFN073892: DSN: [email protected];[email protected]... Invalid route address
2024-06-12T00:17:53.415493-07:00 vCenter sendmail[73892]: 45C7HrFN073892: [email protected], delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=32145, relay=[127.0.0.1] [127.0.0.1], dsn=4.4.1, stat=Deferred: Connection refused by [127.0.0.1]
2024-06-12T00:18:01.979815-07:00 vCenter CROND[74846]: (root) CMD ( test -x /usr/sbin/vpxd_periodic && /usr/sbin/vpxd_periodic >/dev/null 2>&1)
2024-06-12T00:18:01.980049-07:00 vCenter CROND[74845]: (root) CMD (. /etc/profile.d/VMware-visl-integration.sh; /usr/lib/applmgmt/backup_restore/scripts/SchedulerCron.py >>/var/log/vmware/applmgmt/backupSchedulerCron.log 2>&1)
2024-06-12T00:18:01.980190-07:00 vCenter CROND[74847]: (root) CMD ( /usr/lib/applmgmt/syslog/scripts/syslog_cron.sh >/dev/null 2>&1)
2024-06-12T00:18:39.421298-07:00 vCenter sendmail[79704]: 45C7IdfP079704: [email protected], size=1121, class=0, nrcpts=1, msgid=<[email protected]>, relay=root@localhost
2024-06-12T00:18:39.421403-07:00 vCenter sendmail[79704]: 45C7IdfP079704: 45C7IdfQ079704: DSN: [email protected];[email protected]... Invalid route address

Cause

sendmail service was masked and dead, due to which we don't get any email notifications

Resolution

1. Navigate to /var/log/vmware 

   service sendmail status

   If the service is masked, you will receive the message

   sendmail.service
      Loaded: masked (Reason: Unit sendmail.service is masked.)
      Active: inactive (dead)

2. Unmask the sendmail service
    root@XYZ [ /var/log/vmware ]# systemctl unmask sendmail.service
      Removed /etc/systemd/system/sendmail.service.
      Removed /etc/systemd/system/multi-user.target.wants/sendmail.service.

3. Once we unmask the service we get to know the actual status of the sendmail service (active/disabled). So, check the status again

     root@vCenter [ /var/log/vmware ]# service sendmail status
       ● sendmail.service - Sendmail Mail Transport Agent
          Loaded: loaded (/lib/systemd/system/sendmail.service; disabled; vendor preset: enabled)
         Active: inactive (dead)

4.  sendmail service location # /lib/systemd/system ]# cat sendmail.service

5.  If the service is disabled, /lib/systemd/system/sendmail.service; disabled; we can proceed to restart the service

        service sendmail restart

6.  Check the status after restarting the sendmail service

      root@vCenter [ /lib/systemd/system ]# service sendmail status
        ● sendmail.service - Sendmail Mail Transport Agent
           Loaded: loaded 
           Active: active (running) since Wed 2024-06-12 01:47:34 -07; 4s ago
           Process: 71415 ExecStart=/usr/sbin/sendmail -bd -q $QUEUE $SENDMAIL_OPTARG (code=exited, status=0/SUCCESS)
           Main PID: 71417 (sendmail)
           Tasks: 1 (limit: 19660)
           Memory: 1.8M
           CGroup: /system.slice/sendmail.service
                └─71417 sendmail: accepting connections

                    Jun 12 01:47:34 vCenter.XYZ.com systemd[1]: Starting Sendmail Mail Transport Agent...
                    Jun 12 01:47:34 vCenter.XYZ.com sendmail[71417]: starting daemon (8.17.1): SMTP+queueing@01:00:00
                    Jun 12 01:47:34 vCenter.XYZ.com systemd[1]: Started Sendmail Mail Transport Agent.
                    Jun 12 01:47:34 vCenter.XYZ.com sendmail[71417]: STARTTLS=server: file /etc/vmware/vmware-vmafd/machine-ssl.crt unsafe: Permission denied

7.  Create a new alarm definition in the vCenter server and verify if you are getting an email alert.