This document offers steps for a vendor or custom SSL certificate for WCC, Autosys Web Server (AEWS), EEM (iGateway) and CA Directory (dxserver).
NOTE: This assumes that all components are installed on the same machine.
Starting with the WCC .keystore.
#1 Change Directory to wcc, as root user
cd /opt/CA/WorkloadAutomationAE/wcc/data/config
#2 Backup existing keystore
cp -p .keystore .keystore_bkp1
# below are not needed but might help to set JAVA_HOME and PATH to have Autosys Java stuff:
#Unix
JAVA_HOME=/opt/CA/WorkloadAutomationAE/jre; export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH; export PATH
which keytool
#Windows
set JAVA_HOME="C:\Program Files\CA\WorkloadAutomationAE\jre"
set PATH=%JAVA_HOME%\bin;%PATH%
#Note: If Autosys installation Drive is not C: drive, change the drive to that drive in the command prompt, example: D: <hit enter> to change to that drive
#3 List out what currently exists in the keystore
keytool -keystore .keystore -storepass changeit -list -v
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: tomcat
Creation date: Jul 14, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=$COMPUTER_HOST_FQDN$, OU=WCC, O=CA
Issuer: CN=$COMPUTER_HOST_FQDN$, OU=WCC, O=CA
Serial number: 5b0d8902
Valid from: Tue Jul 14 16:45:56 EDT 2020 until: Sun Jul 14 16:45:56 EDT 2030
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 1024-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XX XX XX XX XX XX XX >....x..>.D..ag.
0010: 90 D3 E2 4A ...J
]
]
*******************************************
*******************************************
#4 delete reference to existing private key
keytool -delete -alias tomcat -keystore .keystore -storepass changeit
#5 Ensure the alias got deleted by listing out:
keytool -keystore .keystore -storepass changeit -list -v
Keystore type: jks
Keystore provider: SUN
Your keystore contains 0 entries
#6 generate a new private key / Self signed cert.
#6a) make sure the dname and Subject Alternate Name, both have a value.
keytool -genkey -alias tomcat -keyalg RSA -keystore .keystore -storepass changeit -keypass changeit -keysize 2048 -dname cn=MyWcc-Server.example.com -ext san=DNS:MyWcc-Server.example.com -validity 365
#6b) If there are couple of servers under a load balancer, it is preferred to have all the server names and the real URL name to be used too. Take an example where WCC is accessed via wcc.example.com as the URL, but there are 2 underlying servers, wcc-server1 and wcc-server2, so, the request should have -ext san=dns:wcc.example.com,dns:wcc-server1.example.com,dns:wcc-server2.example.com ).
Example:
keytool -genkey -alias tomcat -keyalg RSA -keystore .keystore -storepass changeit -keypass changeit -keysize 2048 -dname "cn=wcc.example.com,O=Some Org Inc,L=San Jose,S=California,C=US" -ext "SAN=DNS:wcc.example.com,DNS:wccserver1.example.com,DNS:wccserver2.example.com" -validity 365
#6c) The above command does not return anything to the console unless there is an error of some sort
#7) list out what exists in the keystore now:
keytool -keystore .keystore -storepass changeit -list -v
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: tomcat
Creation date: Apr 12, 2022
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=MyWcc-Server.example.com
Issuer: CN=MyWcc-Server.example.com
Serial number: 439ecd7
Valid from: Tue Apr 12 12:25:46 EDT 2022 until: Wed Apr 12 12:25:46 EDT 2023
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: MyWcc-Server.example.com
]
#2: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XX XX XX XX XX XX .....i?..Ab.,Xh.
0010: 94 A5 8B 64 ...d
]
]
*******************************************
*******************************************
#8 Create a Certificate Request (CSR) based off the above private key
#8a) If there are couple of servers under a load balancer, it is preferred to have all the server names and the real URL name to be used too. Take an example where WCC is accessed via wcc.example.com as the URL, but there are 2 underlying servers, wcc-server1 and wcc-server2, so, the request should have -ext san=dns:wcc.example.com,dns:wcc-server1.example.com,dns:wcc-server2.example.com )
keytool -certreq -alias tomcat -keyalg RSA -keystore .keystore -storepass changeit -file wcc.cert.req.csr -ext san=dns:MyWcc-Server.example.com
# NOTE: The above command creates a file wcc.cert.req.csr file in the same folder where we are running all the above commands
# NOTE: CSR validation can also be done, copy the contents of wcc.cert.req.csr to a website like: https://www.sslshopper.com/csr-decoder.html
It should show correct Common Name and SAN names used in the above command.
If they are not correct, then the Cert Request or CSR request was made incorrectly. Repeat ALL the steps from step1 again.
#9 The result of the above, wcc.cert.req.csr needs to be provided to Cert Authority.
#10 Certificate Authority provides a response to the above, usually in the form of a zip file / .p7b file / or individual files (containing the server certificate and root / intermediate certificates)
# NOTE: Order of the certificates preferred for import is, Root (root.crt) first, Intermediate (inter.crt) next, real server certificate (MyWcc-Server.example.com.crt) at the end. These files have to be uploaded to /opt/CA/WorkloadAutomationAE/wcc/data/config folder
#11) As we generated CSR using .keystore and alias tomcat with in that, the Reply that Cert Authority sent is only valid for usage in that keystore + alias=tomcat. So, backup of the keystore
cp -p .keystore .keystore_before_cert_import
#12) Import Root first
keytool -importcert -alias RootCA -file root.crt -keystore .keystore -storepass changeit
Owner: CN=MyCustomROOT-cert, DC=example, DC=com
Issuer: CN=MyCustomROOT-cert, DC=example, DC=com
Serial number: 12345XXXXXXXXXXXXXXXXXXXXXX
Valid from: Fri Mar 06 10:58:36 EST 2020 until: Thu Mar 06 11:08:36 EST 2025
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 1.3.6.1.4.1.311.21.1 Criticality=false
0000: 02 01 00 ...
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
DigitalSignature
Key_CertSign
Crl_Sign
]
#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XX XX XX XX XX ..*.....\.2.....
0010: 05 C7 40 94 ..@.
]
]
Trust this certificate? [no]: yes
Certificate was added to keystore
#13) Import intermediate Cert, the output maybe slightly different from below command, unless there is an error
keytool -importcert -alias intermediateCA -file inter.crt -keystore .keystore -storepass changeit
#13a) If there are more certificates, example, Issuing certificate etc., import them too, we need the full chain and cannot miss any
#14) Finally, import the server certificate
keytool -importcert -trustcacerts -file MyWcc-Server.example.com.crt -alias tomcat -keystore .keystore -storepass changeit
Certificate reply was installed in keystore
#14a) NOTE: the above line is a response to the keytool command, indicating that the reply from Certificate Authority was installed properly
#14b) If there are any other messages, that could potentially mean that the import was not correct
#15) list again, tomcat alias should still a PrivateKeyEntry and contains additional certificates because of the import above
keytool -keystore .keystore -storepass changeit -list -v
Keystore type: jks
Keystore provider: SUN
Your keystore contains 2 entries
Alias name: rootca
Creation date: Apr 12, 2022
Entry type: trustedCertEntry
Owner: CN=MyCustomROOT-cert, DC=example, DC=com
Issuer: CN=MyCustomROOT-cert, DC=example, DC=com
Serial number: 12345XXXXXXXXXXXXXXXXXXXXXX
Valid from: Fri Mar 06 10:58:36 EST 2020 until: Thu Mar 06 11:08:36 EST 2025
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 1.3.6.1.4.1.311.21.1 Criticality=false
0000: 02 01 00 ...
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
DigitalSignature
Key_CertSign
Crl_Sign
]
#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XX XX XX XX XX XX ..*.....\.2.....
0010: 05 C7 40 94 ..@.
]
]
*******************************************
*******************************************
Alias name: tomcat
Creation date: Apr 12, 2022
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: cn=MyWcc-Server.example.com
Issuer: CN=MyCustomROOT-cert, DC=example, DC=com
Serial number: 12345XXXXXXXXXXXXXXXXXXXXXX
Valid from: Tue Apr 12 12:23:14 EDT 2022 until: Thu Apr 11 12:23:14 EDT 2024
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 1.3.6.1.4.1.311.21.10 Criticality=false
0000: 30 0C 30 0A 06 08 2B 06 01 05 05 07 03 01 0.0...+.......
#2: ObjectId: 1.3.6.1.4.1.311.21.7 Criticality=false
0000: XXXXXXXXXXXXXXXXXXXXXX -.%+.....7.....
0010: XXXXXXXXXXXXXXXX ...s...<.......(
0020: 6D 83 95 C1 1C 87 ED CA 4D 02 01 64 02 01 06 m.......M..d...
#3: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
[
accessMethod: caIssuers
accessLocation: URIName: ldap:///CN=CN=MyCustomROOT-cert, DC=example, DC=com,CN=AIA,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=CN=MyCustomROOT-cert, DC=example, DC=com?cACertificate?base?objectClass=certificationAuthority
]
]
#4: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: XX XXXXXXXXXXXXX ..*.....\.2.....
0010: 05 C7 40 94 ..@.
]
]
#5: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
[DistributionPoint:
[ URIName: ldap:///CN=CN=MyCustomROOT-cert, DC=Example, DC=com,CN=AIA,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=CN=MyCustomROOT-cert, DC=Example, DC=com?cACertificate?base?objectClass=certificationAuthority]
]]
#6: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
serverAuth
]
#7: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Key_Encipherment
]
#8: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: MyWcc-Server.example.com
]
#9: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XXXXx XXXXXXXXXXXXX XX .....i?..Ab.,Xh.
0010: 94 A5 8B 64 ...d
]
]
Certificate[2]:
Owner: CN=MyCustomROOT-cert, DC=example, DC=com
Issuer: CN=MyCustomROOT-cert, DC=example, DC=com
Serial number: 12345XXXXXXXXXXXXXXXXXXXXXX
Valid from: Fri Mar 06 10:58:36 EST 2020 until: Thu Mar 06 11:08:36 EST 2025
Certificate fingerprints:
MD5: <###MD5 checksum>
SHA1: < ### SHA1 Checksum ## >
SHA256: < ## SHA256 checksum ## >
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 1.3.6.1.4.1.311.21.1 Criticality=false
0000: 02 01 00 ...
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
DigitalSignature
Key_CertSign
Crl_Sign
]
#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: XX XXXXXXXXXXX XX ..*.....\.2.....
0010: 05 C7 40 94 ..@.
]
]
*******************************************
*******************************************
#15a) NOTE: The imported server certificate somehow does not associate properly to the tomcat alias, as PrivateKeyEntry, the certificate cannot be used by Tomcat.
Errors maybe a) Alias name tomcat does not identify a key entry, b) No private key
#16) Now this keystore can be used by WCC, restart WCC to see it function. Make sure the .keystore file is read-permitted to the operating system user under which WCC runs
#17) This same keystore can be used for EEM or AEWS too (if these modules are on the same server), if not similar process can be followed and then below steps can help convert the result that is suitable to EEM and AEWS.
#18) for EEM (igateway), we need to convert the .keystore to PKCS12
(This conversion is done in the <WCC_INSTALL>/data/config directory. Moving to the iTechnology directory happens at step #36)
keytool -importkeystore -srckeystore .keystore -srcstorepass changeit -destkeystore .keystore.PKCS12 -srcstoretype JKS -deststoretype PKCS12 -deststorepass changeit
Importing keystore .keystore to .keytore.PKCS12...
Entry for alias intermediateca successfully imported.
Entry for alias rootca successfully imported.
Entry for alias tomcat successfully imported.
Import command completed: 3 entries successfully imported, 0 entries failed or cancelled
#18a) NOTE: list to make sure it is valid:
keytool -keystore .keystore.PKCS12 -storepass changeit -list -v -storetype PKCS12
#19) for AEWS, the above keystore needs to be converted the .keystore to BCKFS
#19a) make sure $AUTOSYS is set properly first
echo $AUTOSYS
keytool -importkeystore -srckeystore .keystore -srcstorepass changeit -destkeystore .keystore.bcfks -srcstoretype JKS -deststoretype BCFKS -deststorepass changeit -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath $AUTOSYS/lib/bc-fips.jar
Importing keystore .keystore to .keytore.bcfks...
Entry for alias intermediateca successfully imported.
Entry for alias rootca successfully imported.
Entry for alias tomcat successfully imported.
Import command completed: 3 entries successfully imported, 0 entries failed or cancelled
#20) list the keystore to make sure
keytool -keystore .keystore.bcfks -storepass changeit -storetype BCFKS -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath $AUTOSYS/lib/bc-fips.jar -list -v
#20a) This file .keytore.bcfks can now be used as $AUTOUSER/webserver/conf/.keystore file for AEWS
cp -p $AUTOUSER/webserver/conf/.keystore $AUTOUSER/webserver/conf/.keystore.bkp
cp -p .keystore.bcfks $AUTOUSER/webserver/conf/.keystore
Restart WCC / Restart AEWS for the above changes to be effective.
#IF the same WCC server also hosts EEM server, then the above keystore can be used for EEM using below steps
#21) Stop iGateway:
/etc/init.d/igatewayd stop
#22) Stop dxserver
su - dsa
dxserver stop all
exit
#23) Backup stuff first, as ROOT
cp -rp /opt/CA/SharedComponents/CADirectory /opt/CA/SharedComponents/CADirectory_backup
cp -rp /opt/CA/SharedComponents/iTechnology /opt/CA/SharedComponents/iTechnology_backup
cp -rp /opt/CA/SharedComponents/EmbeddedEntitlementsManager /opt/CA/SharedComponents/EmbeddedEntitlementsManager_backup
#24) Step#18 already converted JKS to PKCS12, so no need to do conversion, list PKCS12, as ROOT
cd /opt/CA/WorkloadAutomationAE/wcc/data/config
keytool -keystore .keystore.PKCS12 -storepass changeit -list -v -storetype PKCS12
#25) extract the private key, as changeit is the password for the keystore through out in the above commands, provide changeit as the password for the next 3 commands (including the passphrases for the keys)
openssl pkcs12 -in .keystore.PKCS12 -nocerts -out encrypted-private.key
## on newer kernels add -legacy flag to the above command openssl pkcs12 -in .keystore.PKCS12 -nocerts -out encrypted-private.key -legacy
ls -al encrypted-private.key
cat encrypted-private.key
openssl pkcs12 -in .keystore.PKCS12 -clcerts -nokeys -out server-cert.pem
## on newer kernels add -legacy flag to the above command openssl pkcs12 -in .keystore.PKCS12 -clcerts -nokeys -out server-cert.pem -legacy
ls -al server-cert.pem
cat server-cert.pem
openssl rsa -in encrypted-private.key -out itechpoz.key
ls -al itechpoz.key
cat itechpoz.key
#26) It is OK to see the warning about not being able to copy folders, folders are not needed
(also make sure to have the intermediate and rootca cert files if they are required in this environment)
cp -p /opt/CA/WorkloadAutomationAE/wcc/data/config/* /opt/CA/SharedComponents/CADirectory/dxserver
chown dsa /opt/CA/SharedComponents/CADirectory/dxserver/*
#26a) NO need to recursively chown in dxserver, just the few files that were copied from the above step.
That's why -r is not present. dxserver/bin/dxadmind and dxserver/bin/dxserver have a sticky bit and have to be owned by root.
Or else CA Directory won't start. DO NOT change this.
#27) change to dsa user
su - dsa
cd $DXHOME
pwd
/opt/CA/SharedComponents/CADirectory/dxserver
#28) start the steps now
cd /opt/CA/SharedComponents/CADirectory/dxserver/config/ssld
#29) list all certs
dxcertgen listca
# remove the default rootca
# in my case the certnumber was 0
# dxcertgen -r 0 removeca
# dxcertgen -r 1 removeca
# dxcertgen -r 2 removeca
# list all
dxcertgen report
#30) Copy the itechpoz.key to the ssld folder first
cp -p /opt/CA/SharedComponents/CADirectory/dxserver/itechpoz.key /opt/CA/SharedComponents/CADirectory/dxserver/config/ssld/itechpoz.key
#31) Import the server cert now
dxcertgen -D itechpoz -n /opt/CA/SharedComponents/CADirectory/dxserver/server-cert.pem certmerge
Example output from a successful run looks like:
! Loading certificate key from /opt/CA/SharedComponents/CADirectory/dxserver/server-cert.pem ...
! Loading private key from /opt/CA/SharedComponents/CADirectory/dxserver/config/ssld/itechpoz.key ...
! Private key matches supplied certificate
! Creating Personality ...
Done.
#32) Import root now
dxcertgen -n /opt/CA/SharedComponents/CADirectory/dxserver/root.crt importca
Example output from successful run of the above command:
! Loading certificate from /opt/CA/SharedComponents/CADirectory/dxserver/root.crt ...
! Writing certificate to trusted.pem /opt/CA/SharedComponents/CADirectory/dxserver/config/ssld/ ...
Writing root certificate to trusted.pem...
Done.
#33) import intermediate
dxcertgen -n /opt/CA/SharedComponents/CADirectory/dxserver/inter.crt importca
Example output from successful run of the above command:
! Loading certificate from /opt/CA/SharedComponents/CADirectory/dxserver/inter.crt ...
! Writing certificate to trusted.pem /opt/CA/SharedComponents/CADirectory/dxserver/config/ssld/ ...
Writing root certificate to trusted.pem...
Done.
#33a) If there are more certificates, for example, Issuing certificates etc., import them too. Full chain is needed and cannot miss any
34) Start Directory
dxserver start all
#35) test using openssl verify the correct certificate is in place now
openssl s_client -connect servername:509
#36) as root user now, copy the keystore.PKCS12 as keystore.p12 into iTechnology folder
https://knowledge.broadcom.com/external/article?articleId=10727
#as root
cd /opt/CA/SharedComponents/iTechnology
cp -p /opt/CA/WorkloadAutomationAE/wcc/data/config/.keystore.PKCS12 keystore.p12
cp -p igateway.conf igateway.conf.bkp
vi igateway.conf
<Connector name="defaultport">
<port>5250</port>
<mustlisten>true</mustlisten>
<conntype/>
<conntimeout>120</conntimeout>
<peektimeout>30</peektimeout>
<maxconnections>1000</maxconnections>
<maxrequestbytes>10000000</maxrequestbytes>
<maxpiperequests>10</maxpiperequests>
<maxAcceptRate/>
<certType/>
<certURI/>
<certPW/>
<keyURI/>
<keyPW/>
<secureProtocol/>
<cipherlist/>
</Connector>
#37) Set certType to p12
Add pr update the below to the <Connector name="defaultport"> section
<certType>p12</certType>
#38) Set certURI to the .p12 certificate filename
Add pr update the below to the <Connector name="defaultport"> section
<certURI>keystore.p12</certURI>
#39) Save the file and existing
#39a) run configtool to encrypt the keystore password into igateway.conf
./ConfigTool -munge -version 4.7.6.1 -comp igateway -conf igateway.conf -tag "TransportReceiver=HTTP;Connector=defaultport;certPW;" -passwd changeit
Example output from successful run from above command:
Operation Successful!!
#40) Start iGateway
/etc/init.d/igatewayd start
#40a) check the EEM URL now and verify the certificate is correct
### Repeat the above on EEM node 2
### Remove and Re-establish EEM HA
### Rengerate Autosys and WCC certs with EEM again - https://knowledge.broadcom.com/external/article?articleId=9957
#41) Restart WCC services
To troubleshoot SSL certificate errors on the browser, the best approach is to use Chrome -> Developer Tools -> Security tab and look at why the errors are happening.
- Is it because the certificate name is incorrect?
- Subject Alternate Names is missing?
- Older ciphers being used?
- Is it just a self-signed certificate that is showing up?
Below is a link to Keystore explore that lets you use a Graphical tool to review your keystore:
https://keystore-explorer.org/downloads.html
Note: Make sure that if you set a password for the Private Key that it matches the Keystore. These can not be different.