New and Updated Packages are very slow to download to a primary Package Server.

Products

• IT Management Suite

Issue/Introduction

New and updated packages take a long time to be downloaded by the Package Servers. Sometimes the package shows up in the package list. Sometimes it doesn’t. After an unexpectedly long time (sometimes hours), the package finally downloads, and sends up its download codebases.

Environment

ITMS 8.x

Deployment Solution 8.x (with a lot of large image packages)

Cause

The Package Server is also functioning as an imaging server, or a server that captures images of other computers.

When it captures an image of another computer it creates a package object which contains (in its "State" column) an entry pointing to the package source. It would be similar to the following excerpt.

<source type="external" location="\\SS-W2K8-01.EPM.local\PkgSvrHostC$\{117e01ad-1afb-4b4b-855f-5e5f746f4ea2}\cache" sourceServer="1182F475-B0BF-4A8F-B7A3-C3D0AC2B6197" isReplicating="false" />

The element also tells the Package Server Agent, when it refreshes its packages, to verify the files related to the image package(s). It does this by hashing each file. This is where the slow-down occurs. Some image packages are made up of many 2GB+ files. Each file can take up to 20 minutes to calculate the hash, and the PSA does this every time it refreshes.

In some cases there were 12 packages and each package had ten *.gho or *.ghs files that were 2-4GB in size. As a result the PSA could spend several hours hashing out all of the image package files, and during that time all new or modified packages are left alone until it is ready to work on them.

The way to tell if it is doing this is to search the logs of the Package Server agent for " AddDirectoryFiles: hashed file ".  There will be many entries similar to the following:

Process: AeXNSAgent.exe (5080)
Thread: 7036
Module: AeXPackageDelivery.dll
Source: Snapshot
Description:  AddDirectoryFiles: hashed file: '\\SS-W2K8-01.EPM.local\PkgSvrHostC$\{117e01ad-1afb-4b4b-855f-5e5f746f4ea2}\cache\Windows_7_x64_sysprep004.ghs'  hash='13I7jrTfyzIvhmYZUr6ryY67CAwMN7npIAY/BTU4vCk='

Also, if the query attached to this article “Site Server Owned Image Packages.sql” is run against the database it will identify all of the Imaging package objects, and what Package Server owns them.

Resolution

There is no way currently to speed up the hashing of image files.  

There are two ways to workaround this problem.

1. 1. Stand up another Package Server in the same site as the current Package Server. The new server will not be responsible for capturing images, therefore it will not be concerned with calculating the hashs for the image packages because it does not own them. It  will only be concerned with downloading new and updated packages, which it will be able to do much quicker.
2. 2. Another option would be to re-import the image file to the SMP using instructions provided in HOWTO93711. This will alleviate the package server from the duties of maintaining the image files, and concentrate only on downloading and maintaining the files for all packages (which don't require heavy duty hash checking).

NOTE: Once the .img or .gho files have been imported to a specified folder on the SMP you can find and delete the old package object(s) via the SMP console by going to "Settings > All Settings > Deployment > Disk Images"