Cert overwritten in agent logs

Products

• IT Management Suite

Issue/Introduction

There are informational entries in the agent logs and you would like to know what some of it means.  For example:

Informational logging that says the certificates were restored.

Some certificates were overwritten in "Intermediate Certification Authorities' store. 

A Certificate was overwritten in the ' Trusted Root Certification Authority" 

Periodic certificate validation started.
Failed to restore the certificates...
Validating certificate store...

How does this process of certificate management work?

 

Environment

Release : 8.x

Resolution

Certificate integrity validation

The current system provides some integrity mechanisms – every time SMA adds or modifies any certificate it also performs Windows certificate store backup. The backup copy is saved into SMA secure storage in "AgentCore\CertificateStoreBackup\N" folders where N is the internal certificate store ID. Certificate store backup copy is stored as a binary blob, no private keys are stored in there. Later when SMA needs to perform some certificate related operations like agent registration, CEM certificate retrieval, site server certificate retrieval it first validates if Windows certificate stores have all SMA certificates from the backup. If some certificate is missing SMA first restores them from the backup copy. SMA also performs the periodic certificate validation – every 3 hours SMA compares the backup copy of the certificate stores with the actual certificate stores and restores the missing or modified SMA certificates. The same validation is performed during SMA startup. This scenario is important for Windows 10 upgrade scenarios where Windows 10 upgrade does not migrate the content of SMA server personal store "AeXNSClient\Personal", in this scenario the store content will be restored by SMA upon startup after Windows 10 upgrade is completed.

The whole certificate store backup is replaced by individual certificate backup. The backup location is "AgentCore\CertificateBackup\N\thumbprint", where N is the store ID and "thumbprint" is the actual certificate thumbprint.

Certificate Stores used by SMA

SMA installs various certificates in different stores, specifically:

• Agent and gateway CEM certificates go into "AeXNSClient\\Personal" store, which is store associated with "Symantec Management Agent" service.
• IIS binding certificates on site server go into "My Computer\\Personal" store. Previous version of SMA copied site server certificates into "Trusted Root Certification Authority" as well.
• CA certificate got into either "Trusted Root Certification Authority" or "Intermediate Certificate Authorities" store depending if it signed or not. "Trusted Root Certification Authority" store should never contain a signed certificate, this can break IIS authentication, and clients will not be able to connect to that IIS using HTTPS. The previous version of SMA can copy signed certificate into "Trusted Root Certification Authority" store, 8.1 RU2 upgrade should take care about the problem and move signed certificates into "Intermediate Certificate Authorities" store.
• CA self-signed certificates also go into "Client Authentication Issuers" store on site server to allow IIS on Windows 2012 or later to correctly authenticate the clients. Previous version of SMA also copied signed certificates into "Client Authentication Issuers" store, this is not correct and 8.1 RU2 should take care about that and remove needless certificates copies.

No other stores are used by SMA.

Regarding the questions.  Remember that these are informational messages and there are no problems at all. 

• 0 certificates were restored, 4 certificates were overwritten in 'AeXNSClient\Personal' store <- means no certificates are missing, 4 certificates are the same but they were overwritten to fix extended properties in case they are missing or were changed by 3rdparty software,. The content of the certificate itself is not changed, only extended properties which are not part of "security part" of certificate are modified.
• Failed to restore the certificates to 'Local Computer\Personal' store, no backup data, error: The system cannot find the file specified (0x00000002) <- there were nothing to restore in Personal store, error 2 says that. You can use communication profiles to distribute root, intermediate or end certificates and they are placed in the appropriate stores - Root, Intermediate or Personal.

Regarding SMA service private store - only CEM certificates are stored there. This is just an additional security step to isolate those certificate from accidental removal. If machine is in CEM mode and those certificate removed then machine would loose connection to NS. This store is used since the very first release of CEM.