HTTP Status 404 Not Found On sis-ui Rest API Response

Products

Data Center Security Server Advanced

Issue/Introduction

Symantec DCS (Data Center Security) relies on rest APIs (application programming interface) to receive and respond to requests made via the UMC (Unified Management Console), Java Console (also known as the policy editor), and other API calls made directly to the web-based listener service. This article covers the application layer of communication. For issues related to presentation and session layer issues, or lower, refer to the following articles:

Transport layer (TCP): https://knowledge.broadcom.com/external/article?legacyId=TECH237122
Presentation layer (SSL client/server trust): https://techdocs.broadcom.com/us/en/symantec-security-software/endpoint-security-and-management/data-center-security-(dcs)/6-9/installing-upgrading-v127913953-d3608e151987/installing-the-unified-management-console-root-ca-v127944020-d3608e165515.html

Environment

Symantec Data Center Security (DCS) Manager

Cause

There are several potential causes to this error, however, we will cover two of the most common causes and their respective solutions. They are:

Symantec Data Center Security Server Manager services started too soon, after a server reboot.
Java keystore (JKS) file does not have the necessary trust when posting/reading to and from 4443*.

*4443 is the default port used by DCS for API communications. Although this issue is technically on the presentation layer (and on up) it presents as an application-level error when the call is made from UMC or the Java Console.

A backup of all the DCS Manager files, and its database should be performed regularly, this article implies that regular backups of the DCS Manager(s) and database have taken place. For details on how to perform a full backup see: https://knowledge.broadcom.com/external/article/159322/backup-and-restore-dcs-database-for-disa.html

Resolution

Premature Initiation of Symantec Data Center Security Server Manager Services Following Server Reboot

For the services starting too soon issue, there is a quick resolution: Simply restart the DCS Manager services followed by the two UMC services:

In order to mitigate for this potential issue, consider setting the following services from the Microsoft Services (services.msc) to "Automatic (Delayed Start)"...

Symantec Data Center Security Server Manager
Symantec UMC Credential Service
Symantec UMC Telemetry Service

For example:

Note: This mode means the service will start after a short wait for other Windows services marked as Automatic.

Java Keystore (JKS) File Trust Issues in Posting/Reading to/from Port 4443

Since there are several 3rd party articles which cover the mechanics of adding a trusted CA (certificate authority / self-signed / certificate) to a JKS (and other keystore types), we will cover how to identify and extract the necessary certificate(s) from the PKCS#12 (Public Key Cryptography) utilized by DCS, including step-by-step instructions on how to add these to the trusted keystore used by the JRE (Java runtime environment) installed with DCS.

Note: DCS names the extensions for its PKCS12 stores from the standard ".pfx" or ".p12" to ".ssl". The only exception to this is the produced agent-cert.ssl which is a BASE64 encoded issued certificate, and does not contain a private key or keystore indexing. For the remaining steps of this article we will create a folder named "Example" in the "C:\" drive (C:\Example). Any name(s) may be used, just remember to modify your actual commands accordingly.

Copy the ssl.ssl file from [DCS Install Directory]\server\ into "C:\Example"
For example: C:\Program Files (x86)\Symantec\Data Center Security Server\Server\sss.ssl copied "C:\Example"
Note: If an IPS (prevention/hardening) policy is in place, copying/accessing this file may not be possible. Consider this, and do the needful via policy, driver, or Security Group to allow access. The following steps are performed on the DCS Manager server itself. Consider your data sensitivity policies when accessing/modifying any certificate(s).
Copy the server.xml file from [DCS Install Directory]\Server\tomcat\conf onto the desktop or other directory.
For example: C:\Program Files (x86)\Symantec\Data Center Security Server\Server\tomcat\conf copied to the Desktop.
Open the server.xml and take note of (copy) the certificate keystore password identified by the certificateKeystorePassword directive for the SSS-Console-Service.
For example:

Note: For this example, the following working folder will be created and used: "C:\Example". A fictitious PKCS12 keystore file will be used, the password will be obscured or modified. The keystore passwords are randomly created during installation.
With the password noted (copied), open an administrative CMD (Command Prompt) console. CD (change directory) to the following location: "[DCS Install Directory]\Server\jre\bin".
For example:
From the CMD enter the following:
```
cd C:\Program Files (x86)\Symantec\Data Center Security Server\Server\jre\bin
```

Use the Java keytool.exe command-line (CLI) tool to identify CA (issuer) certifciate file(s) from the sss.ssl keystore, using a command like:

C:\Program Files (x86)\Symantec\Data Center Security Server\Server\jre\bin>keytool.exe -keystore "C:\Example\sss.ssl" -storepass "CdlxmLPSnCng65ttkeXXXtWhAS4fIlfuRlanOXxc" -storetype PKCS12 -list

Take note of the keystore alias name. It can be identified by either "sss" (which is the default installation alias) or any single-line value after "Your keystore contains 1 entry" before the following "," (comma) and date.
For example:

Note: In the above example the alias entry is "sss". If a thirdparty or in-house issued certificate is used, the alias may be autogenerated by the issuer.

Using the alias value from the previous step, extract the issuer certificate with a command like:

keytool.exe -keystore "C:\Example\sss.ssl" -storepass "CdlxmLPSnCng65ttkeXXXtWhAS4fIlfuRlanOXxc" -storetype PKCS12 -list -alias sss -rfc

Copy the contents of "Certificate[2]" from (and including) "-----BEGIN CERTIFICATE-----" to (and through) "-----END CERTIFICATE-----"
Note: If there is only 1 certificate listed, then perform this step for "Certificate[1]" as it is self-signed.
For example:
Paste the copied text into a notepad file and save as root1.cer. Important: Ensure that the "Save as type:" is set to "All Files" and the "File name" is "root1.cer" with the ".cer" extension.
For example:
Repeat these steps for any other certificate(s) listed except for the first, naming them sequentially (root2, root3, etc). Note: In cases when an in-house or 3rd party certificate is issued, there may be a Certificate [3], Certificate [4] (or more) listed. These represent the chain of authority for the issued certificate which is "Certificate [1]", extending to the root or CA. Any and all of these must be added to the JKS file (cacerts) using the following steps (for each).
Now that the root (otherwise known as a CA certificate) is extracted, along with any of the certificates in the chain (see previous conditional step), we will now import them into the cacerts JKS certificate store, which is used for Java trust when communicating to the API listener (on 4443 by default). From an administrative command prompt CD to "[DCS Install Directory]\Server\jre\bin".
For example:
```
cd C:\Program Files (x86)\Symantec\Data Center Security Server\Server\jre\bin
```

From this directory run the following commands for each of the saved root#.cer files, update the alias variable to match the name, with a command like:

keytool.exe -keystore "C:\Program Files (x86)\Symantec\Data Center Security Server\Server\jre\lib\security\cacerts" -storepass "CdlxmLPSnCng65ttkeXXXtWhAS4fIlfuRlanOXxc" -storetype JKS -import -file "C:\Example\root1.cer" -alias root1

Enter yes when prompted to trust the certificate.

For example:

Once added into the cacerts file, restart the following DCS services in the order listed:
Symantec Data Center Security Server Manager
Symantec UMC Credential Service
Symantec UMC Telemetry Service

Note: Allow sufficient time for the services to start.

Important: If there are more than one DCS FE (front-end) Manager in play, you may simply copy the cacerts file from the primary Manager (or whichever the above steps were performed on) and overwrite its existing cacerts file (make a copy of the original as a backup).

Additional Information

This issue presents very similar to the known issue of browser certificate trust, presented with the following message:
System Error: Unable to connect to DCS: Server. Ensure that you have accepted the DCS: Server certificate in the web browser. If the problem persists, contact service administrator.

A quick method to distinguish this issue from the common browser trust issue, is to run Chrome with the --ignore-certificate-errors flag. Important: This troubleshooting method should ONLY be used as a troubleshooting isolation step, and should NOT be used to circumvent browser security.