Salt Minion Presence shows "Disconnected"

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• When viewing Minions in Aria Automation Config / Tanzu Salt Portal > Targets > Minions > The Minion Presence is marked as 'Disconnected'. 
  

Environment

Aria Automation Config 8.x

VMware Tanzu Salt 8.x

Cause

• The "Disconnected" state can be caused by:
  • Wrong status in Salt Database
  • Network issue between Salt-Master and Salt-Minion.
  • Issue with salt minion service.
  • Issues with salt master communicating to Raas server. 
    • Master service unhealthy.
  • SSEAPE Plugin / Salt Master Plugin incompatible with Raas server instance or is not installed correctly.
    • Salt Master fails to publish events to Raas server. 

Resolution

1. Validate that the salt minion server being targeted is up and the Salt minion service is running healthily:
   systemctl status salt-minion,service
   • If the salt-minion service is not running healthily or stopped, restart the salt-minion service.
     systemctl stop salt-minion.service
systemctl start salt-minion.service
   • If the salt minion service still fails to start, enable debug logging on the salt minion service and validate the /var/log/salt/minion file.
     Refer How to Enable debug logging on SaltStack and VMware Aria Automation Config. 
2. Validate that the salt-master service is up and running healthily.
   • systemctl status salt-master.service
     • If the salt master service still fails to start, enable debug logging on the salt master service and validate the /var/log/salt/master file.
       Refer How to Enable debug logging on SaltStack and VMware Aria Automation Config.
   • On the Raas UI (Aria Automation Config / Tanzu salt Portal) Navigate to Administration > Master Keys > Validate that the master key is accepted.
   • On the Raas UI (Aria Automation Config / Tanzu salt Portal) Navigate to Administration > Master Plugin > Validate that the master plugin is marked updated/ connected. 
3. Validate that the Raas service is running healthily.
   systemctl status raas.service
   • If the raas service is not running healthily or stopped, restart the raas service.
     systemctl stop raas.service
systemctl start raas.service
   • If the raas service still fails to start or complains about errors, enable debug logging on the raas service and validate the /var/log/raas/raas file.
     Refer How to Enable debug logging on SaltStack and VMware Aria Automation Config. 
4. Test TCP connection from Salt-Minion to Salt-Master's port 4505/4506.
5. From the salt master command line - initiate a test.ping to the minion:
   salt <minion-id> test.ping
   • Validated if the job executes successfully, i.e. it captures a return - success : true or false and is able to publish the same on the raas UI : Aria Automation Config > Activity > Jobs > see if the job is listed and validate it's status.
   • If the salt job fails to execute or returns success: false - validate salt master and salt minion debug logs. 
6. From the Raas UI execute "test ping" and validate if the presence status can change from "Disconnected" to "Present":
   • Aria Automation Config UI > Targets > Minions > Select the minion instance and click on Run Job.
     
   • Search for job: test.ping 
     
   • Select Run Job.
   • Validate the status of the job created. 
   • If successful, Refresh the UI and verify if the minion presence is not marked Present. 
7. If the presence status is still "Disconnected", re-attempt to enable Presence:
   • Aria Automation Config UI > Targets > Minions > Select the minion instance and click on Run Job.
   • Search for job: enable.presence. 
   • Select Run Job.
   • Validate the status of the job created. 
   • If successful, Refresh the UI and verify if the minion presence is not marked 'Present'. 
8. If the issue still persists, please raise a support request with Broadcom Support Team.