Agent Upgrade Fails Due to Presence Of Reg Keys From Previous Versions

Products

Carbon Black App Control (formerly Cb Protection)

Issue/Introduction

Agent Upgrade error returned to Console similar to:

Agent upgrade: Failed executing Installer[C:\ProgramData\Bit9\Parity Agent\ParityHostAgent.msi] 
Cmdline[/qN /i "C:\ProgramData\Bit9\Parity Agent\ParityHostAgent.msi" /l*v+ "\\?\globalroot\device\harddiskvolume3\programdata\bit9\parity agent\logs\ParityHostAgentInstall_TIMESTAMP.log"]. Fatal error during installation.

The ParityHostAgentInstall_TIMESTAMP.log includes errors such as:

B9_UPGRADING_BLOCKED = {PRODUCT GUID}
(FAILURE) Agent cannot be installed per-user.
Product: Bit9 Agent -- Error 1714.The older version of Bit9 Agent cannot be removed. Contact your technical support group. System Error 1612.

A manual install fails with an error similar to

Upgrade from <VERSION> to <VERSION> is not allowed

When reviewing the Agent upgrade log the following RegEx search returns multiple Product GUIDs being logged:
```
DA971CA3|9F2D4E59|1C327257|80947C61|38333681|253ED65C|49E143F1|AC01D403|B5B27F36
```
- These are partial GUID values for each Major Version.
- Only two GUIDs (current version and the upgrade version) should be returned.
- If more than two are returned, registry cleanup is required.

Environment

App Control Windows Agent: All Supported Versions

Cause

Summary:

A pre-installation check has found registry keys associated with multiple Product GUIDs for the Agent.

What Causes Multiple Registry Keys

The workflow that causes this situation typically follows similar to the following:

App Control Agent is initially installed on the endpoint (ex: Version 8.8.2).
- Windows Installer creates a copy of the installation media locally in either of the following paths:
```
C:\Windows\Installer\
C:\Windows\ccmcache\
```
Installation media is later removed (or not persisted during SCCM deployment).
First Major Version Agent Upgrade
1. Agent is upgraded to a new Major version (ex: Version 8.9.6).
2. During upgrade, Windows Installer attempts to reference the previous installation media for removal of old files.
3. Previous installation media is no longer available and a "best effort" by Windows Installer is made for the removal.
4. The upgrade task completes successfully, but with left over registry keys due to the "best effort" upgrade by Windows Installer.
  - Health Checks are generated by the Agent for FailureId 1010 - multiple GUID in registry
Second Major Version Agent Upgrade
1. A subsequent Agent upgrade is attempted to a new Major version (ex: Version 8.10.2)
2. Pre-installation check finds Registry Keys left over from the first Major Upgrade.
3. Multiple installations of the Agent on the same endpoint are not permitted, the upgrade fails.

Resolution

Verify the Presence of Leftover Registry Keys

Launch an administrative command prompt

Verify the file version of the Agent that is installed:

"C:\Program Files (x86)\Bit9\Parity Agent\DasCLI.exe" status

Example output:
Version Information
    CLI:        8.7.8.787 6/17/2022 5:00:17 PM
    Agent:      8.7.8.787 6/17/2022 5:00:17 PM
    Kernel:     8.7.8.787 6/17/2022 5:00:17 PM

Validate only one GUID shows in \UpgradeCodes\ with the use of the following command (or use Regedit to manually navigate to the path)
```
reg query HKEY_CLASSES_ROOT\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9
```
The Registry Key should contain a single Package Code, matching the Agent version installed on the system. Example:
- Query output example of multiple Package Codes returned (versions 8.7.8 and 8.6.x):
```
HKEY_CLASSES_ROOT\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9
    C56DE352F399D2544A140184D1CFDFA9    REG_SZ
    95E4D2F9825022B46B466A0B8B4B28EE    REG_SZ
```
- GOOD registry shows only the Package Code for the currently installed Agent 8.7.8: C56DE352F399D2544A140184D1CFDFA9
- BAD registry shows multiple Package Codes for the current, and older agent versions
WARNING: If none of the Package Codes in this registry path match the current Agent version listed in "dascli status" then the Agent registry keys have been corrupted

Although the Agent is fully operational, it will not be possible to upgrade it until it has been cleaned using the Agent Uninstall Tool ONLY.
In some instances, registry entries may exist in Keys outside of \UpgradeCodes\.
- In those instances additional steps may be necessary (such as automated cleanup) to remediate the issue.

Cleanup the Registry Keys

WARNING: Do not delete the Registry Keys for the currently installed Agent version, as this will corrupt it, and the Uninstall Tool will be required.

Option 1: Manual or Batch Script Cleanup

Use the steps above to verify which Package Code & Agent Version combination is preventing the upgrade.
Delete the corrupted registry keys from the previous Agent versions in the bellow Registry locations
- Replace PACKAGECODEHERE with the previous agent package code found in Step 1, for example:
```
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Features\95E4D2F9825022B46B466A0B8B4B28EE /f &
```

Run the following commands manually in CMD or save them as a batch script and execute it via GPO or other deployment method:

reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Features\PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Products\PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\Managed\S-1-5-18\Installer\Features\PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\Managed\S-1-5-18\Installer\Products\PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\Managed\S-1-5-18\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\PACKAGECODEHERE /f &
reg delete HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &
reg delete HKEY_CLASSES_ROOT\Installer\Features\PACKAGECODEHERE /f &
reg delete HKEY_CLASSES_ROOT\Installer\Products\PACKAGECODEHERE /f &
reg delete HKEY_CLASSES_ROOT\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &
reg delete HKEY_USERS\.DEFAULT\Software\Microsoft\Installer\Features\PACKAGECODEHERE /f &
reg delete HKEY_USERS\.DEFAULT\Software\Microsoft\Installer\Products\PACKAGECODEHERE /f &
reg delete HKEY_USERS\.DEFAULT\Software\Microsoft\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &
reg delete HKEY_USERS\S-1-5-18\Software\Microsoft\Installer\Features\PACKAGECODEHERE /f &
reg delete HKEY_USERS\S-1-5-18\Software\Microsoft\Installer\Products\PACKAGECODEHERE /f &
reg delete HKEY_USERS\S-1-5-18\Software\Microsoft\Installer\UpgradeCodes\BE699B28D1B16A04D9F1AA3A0C28A1C9 /v PACKAGECODEHERE /f &

Attempt the Agent Upgrade once more.
- In some instances an Agent service restart may be required before the upgrade is successful.

Option 2: PowerShell Script Cleanup

A PowerShell script to assist in the automated cleanup of these registry entries is attached to this article.
It will verify the current Agent version installed, delete registry entries for any unrelated version and log accordingly.
1. The script should be tested only on a small subset of machines first.
2. After downloading the file to the machine it may be required to: Right click the file > Properties > Unblock.
3. Open the script in a text editor and adjust the two variables accordingly
  - $logFile String value of a local path.
  - $testMode Boolean value for logging only or registry value removal (defaults to logging only).
4. This script will require PowerShell version 2.0 or newer (Server 2008 R2/Windows 7 or higher).

Additional Information

If the Agent upgrade was forced through by temporarily disabling Tamper Protection on the endpoint a partial upgrade could occur that will require the Agent Uninstall Utility.
Please check to be sure the SCCM Package is properly configured to persist the files on the endpoint, as outlined in the Agent Installation Guide.
For more information about the SCCM Cache Folder and how to handle MSI installs please check the following external KB.

Attachments

AgentRegClean.zip get_app