Task Server's Object Host Service fails to bind on all ports

Products

• IT Management Suite

Issue/Introduction

Task Server has 2 services that are installed:
Client Data Loader (CDL) and the Object Host Service (objHost). 
CDL usually binds successfully on 50120, but could be blocked. 
Most common is a failure of the objHost to bind on all it's ports: 50121, 50122, 50123, 50124. 
Any one of the 5 ports missing will cause issues with task execution.

To verify if you are having this problem, go to the affected Task Server / Site Server (or SMP Server), open a command prompt, and run the following:

netstat -an

Check to see if you can see all 5 listening ports.  It should look something like this:

0.0.0.0:50120
0.0.0.0:50121
0.0.0.0:50122
0.0.0.0:50123
0.0.0.0:50124

If any of those 5 ports are not listening, this is the symptom we are looking for.

Environment

ITMS 7.x, 8.x

Cause

There are several possible reasons, some of which are still being researched.  These include:

• Another product blocking these ports
• Another product taking the ports first and thus preventing objHost or the CDL from using them
• A problem with the installation of the Task Server (i.e. missing or damaged services)
• Issue where the Remote Task Service is unable to resolve the NS name via ICMP
• SQL Server is busy with other internal tasks causing the requests from the SMP Server to be queued and take longer to process NSE's

Resolution

** Remember, be sure what kind of server you are troubleshooting:

• Only the SMP Server will bind on all 5 ports
• All other Site Servers will only bind on 3 ports
• A remote site server will not bind on ports 50122 & 50123

If you are sure you have a problem, here are some troubleshooting steps you can take to resolve this issue:

1. Check for port blocking.

External systems that may block these ports include firewalls and GPO's that can stop products from binding on certain ports.  If your company has a policy about ports that need to be blocked and/or opened, you should consider two things:

1. Please review the Ports document provided by us to assist you in determining what ports you want to allow, found here: Ports and Protocols for IT Management Suite
2. Consider, where possible, picking more "acceptable" ports.  For instance, on the Task Server settings page, the ports can be changed.

2. Check a 3rd party app using the ports.

Another type of block can be another application.  You may want to see if your server is hosting another application that is taking that port already.  For instance, you may find all those ports listening as shown in the graphic above, but if you run TCPView, you may find that AtrsHost is not listening on 50121, 50122, 50123, and 50124.  This happened for one customer at least.  NetStat will show the ports as open, but TCPView did not show the AtrsHost as using them, indicating a 3rd party app had those ports instead of us.

If you find the ports open but for the wrong application, we can either change the ports for the 3rd party app or modify the ports that the Task Server is using. If you need to modify the ports Task Server uses, those ports are found here: Settings | Notification Server | Site Server Settings, then expand on the left: Site Management\Settings\Task Service\Settings and select Task Service Settings.

3. Verify you are using default ports.

It is not necessary to use the default ports, but if you've attempted to change them (i.e. as suggested above), you may run into another issue found here: Task Server Port Bindings won't change (KB 177074).  If so, please follow the steps in that KB to resolve the issue.

4. Check the taskmanagement.log file to see if the Task Service can communicate with the NS.

We have seen cases where the base NS agent is able to resolve the Notification Server's name while the Task Service is unable to do so. Review the log file under <InstallDir>\Altiris Agent\Client Task Server\logs to see if the Task Service on the Site Server is communicating with the NS

5.  Ensure that the Management Agent is running on the Notification Server

The Symantec Management Agent (Altiris Agent) on the Symantec Management Platform server must be installed and running on the SMP server.

The agent's Altiris Server must be correctly configured to reference the installation of the Symantec Management Platform server.  That is, the FQDN of the localhost.

6. Contact Support.

We are looking into other issues relative to this problem and have additional possible troubleshooting steps if the above do not resolve the issue for you, so please contact us.  We will review all of the above with you if you do, please have the results of your research ready.  Thanks!

7. Verify the installation of Task Server.

We've found that, sometimes, the installation of Task Server itself is the cause of this problem.  Either a service is not installed, or files are missing, or something.  In these circumstances, a reinstallation of the Task Server in question may be in order and has solved the problem for some customers.  You should follow the steps outlined in Repair / reinstall / uninstall a Task Server agent manually (KB 176958) to remove and reinstall the Task Services on that server.  For some customers, after a reinstall, the ports bound properly.

8.  Verify that Notification Server Task Server components are Installed

• Check that the folder C:\Program Files\Altiris\TaskManagement folder exists.
• Check that the file AtrsHost.exe file is the same version as the C:\Program Files\Altiris\Notification Server\Bin\AeXSVC.exe file.
  
  

9.  Verify that the Altiris Object Host Service is using the correct instance.

The Altiris Object Host Service that is installed onto the Symantec Management Platform Server is installed in a different location to those of child Task Servers.  When the service is installed on the Notification Server itself it is configured to run from the C:\Program Files\Altiris\TaskManagement\AtrsHost.exe file.

9.1  To check that the service on the Symantec Management Platform is configured correctly:

• Open the Windows Services console;
• Double-click the Altiris Object Host Service;
• Verify that the Path to executable points to "C:\Program Files\Altiris\TaskManagement\AtrsHost.exe".  In some cases, this has been pointing to C:\Program Files\Altiris\Altiris Agent\Client Task Management\AtrsHost.exe

If this is not the case, you will need to modify the registry directly to update the service's settings.  Please note that modifying the registry incorrectly could leave your system in an unusable state.  We recommend having a current and valid backup of the registry prior to editing.

• Stop the Altiris Object Host Service
• Stop the Altiris Client Task Data Loader service
• Open Registry Editor
• Navigate to:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\atrshost

• Edit the InstallPath REG_EXPAND_SZ value and change it to:

"C:\Program Files\Altiris\TaskManagement\AtrsHost.exe"

• Verify the service settings have changed by checking the service properties as per section 9.1 above
• Start the Altiris Client Task Dataloader service
• Start the Altiris Object Host Service

Verify that the services have bound correctly as per the Diagnosis steps.

Note: All paths mentioned within this article reference a default installation.  If your installation is not to the default location please modify the paths accordingly.

10. Restarting the SQL server and scheduling maintenance

Rebooting the SQL Server, the queues (EvtQFast and EvtQueue primarily) started processing the NSEs

If the SQL Server seems to be working as usual, but you notice that the Altiris Console is acting too slow or slower than before, it is recommended to evaluate the SQL Server performance and how it is maintained (referring to if there is a SQL Server Maintenance Plan put in place as part of the best practices for most companies).