Cannot start CORBA applications with ERROR TRACE at CsCorbaMgr.cc(1256): Exception: CORBA::TIMEOUT
search cancel

Cannot start CORBA applications with ERROR TRACE at CsCorbaMgr.cc(1256): Exception: CORBA::TIMEOUT

book

Article ID: 271114

calendar_today

Updated On: 08-19-2024

Products

Network Observability Spectrum

Issue/Introduction

When starting the Spectroserver, it fails with the following error as seen in the $SPECROOT/SS/VNM.OUT and/or $SPECROOT/LS/LOCSERV.OUT and/or $SPECROOT/SS/DDM/ARCHMGR.OUT files:

Aug 02 09:51:33 ERROR TRACE at CsCorbaMgr.cc(1256): Exception: CORBA::TIMEOUT
        Minor: 1447165953
        Completion Status: NO





The CORBA Naming Service shows the following error in its log

$SPECROOT/bin/VBNS/NAMINGSERVICE.OUT

ExtFactory fails!
org.omg.CORBA.INITIALIZE: Could not initialize java.security.cert.CertificateExpiredException: NotAfter: Wed Aug 02 08:03:51 EDT 2023  vmcid: 0x0  minor code: 0  completed: No
at com.borland.security.core.Init.pre_init(Init.java:672)
at com.inprise.vbroker.orb.ORB.initialize(Unknown Source)
at com.inprise.vbroker.orb.ORB.set_parameters(Unknown Source)
at org.omg.CORBA.ORB.init(ORB.java:353)
at com.inprise.vbroker.naming.ExtFactory.main(Unknown Source)

 

Environment

DX NetOps Fault (SPECTRUM) versions up to and including 21.2.8.

Cause


SpectroSERVER components ( LocServer, SpectroSERVER, ArchMgr) by default bind a CORBA port AND a secure CORBA port. The secure CORBA port requires a valid certificate that by default is provided by Spectrum out of the box.


$SPECROOT/lib/SDPM/partslist/NAMINGSERVICE.idb (Corba Namingservice)
Secure: 14016
Normal: 14006

$SPECROOT/LS/.locrc (LocServer)
Secure: 14014
Normal: 14004

$SPECROOT/SS/.vnmrc (SpectroSERVER)
Secure: 14012
Normal: 14002

$SPECROOT/SS/DDM/.configrc  (ArchMgr)
Secure: 14013
Normal: 14003


The certificates used to establish the secure CORBA ports expired.

Resolution

This is resolved in DX NetOps Fault (SPECTRUM) in versions 21.2.10 and higher.

In versions 21.2.8 and lower, the out-of-the-box certificate expired on Aug 2, 2023.  The certificate is stored in the $SPECROOT/custom/VBNS/identities/spectrum/cert0.

$SPECROOT/custom/VBNS/identities/spectrum
command:
openssl x509 -in cert0 -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 485396356 (0x1cee8f84)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = CA, CN = spectrum-server
        Validity
            Not Before: Aug  2 12:03:51 2021 GMT
            Not After : Aug  2 12:03:51 2023 GMT
        Subject: C = US, O = CA, CN = spectrum-server
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

 

The solution is to either upgrade to 21.2.10 or higher (which will be EOS Dec 31, 2023) or disable secure CORBA on the SpectroSERVER:

If secure CORBA is not enabled on OneClick you can disable the CORBA Security by editing these files on all SpectroSERVER systems (primary and secondary) and change the vbroker.security.disable from false to true:

$SPECROOT/.corbarc
$SPECROOT/.jcorbarc

vbroker.security.disable=true




NOTE: There are also .jcorbrc and .corbrc files in the $SPECROOT. These do not have to be modified. The two files that need to be modified are .corbarc and .jcorbarc.

You will need to start the Naming Service as follows:

  • Log into the system as the user that owns the Spectrum installation
  • If running Windows, start a bash shell by running "bash -login"
  • cd to the $SPECROOT/bin directory and run the following command:

./launchinstdbapp localhost NAMINGSERVICE y NAMINGSERVICE.OUT

  • Verify the nameserv process is running
  • Check the $SPECROOT/bin/VBNS/NAMINGSERVICE.OUT file for the error
  • Verify you are no longer seeing the error in the $SPECROOT/SS/VNM.OUT and/or $SPECROOT/LS/LOCSERV.OUT and/or $SPECROOT/SS/DDM/ARCHMGR.OUT files.

Additional Information

There are two scenarios where the $SPECROOT/custom/VBNS/identities/spectrum/cert0 file can be replaced by an older version:

  1. Restoring the $SPECROOT/custom/ directory from an older backup
  2. Migrating the Spectrum to a new machine and copying over the $SPECROOT/custom/ directory from an older Spectrum version

To overcome it you can simply copy the $SPECROOT/bin/VBNS/identities/spectrum/cert0 file to the $SPECROOT/custom/VBNS/identities/spectrum/ directory to replace the expired Corba certificate file.


**Note**
   If your SpectroSERVER is currently running (has not been stopped since before Aug 2nd) you can modify the .corbarc and .jcorbarc files as noted
      above to avoid failure on restart.


1) Secure Corba
       By default, the SpectroSERVERs will bind both secure and nonsecure CORBA ports. OneClick by default does not have
         secure CORBA enabled. If Secure Corba is enabled it will need to be disabled as well if disabling in the steps above to
         have the Spectrum processes run on the SpectroSERVER (namineservice, locserver, SpectroSERVER, ArchMgr ..etc).


   



2) Please also reference information pertaining to a previous issue with the Naming Service and Secure Corba:

     https://knowledge.broadcom.com/external/article?articleId=238233 

 

3) Secure Corba Certificate Dates for Various Versions

    

21.2.8 & earlier expired Aug 2, 2023
21.2.9 - 21.2.12 expires Jan 24, 2024
22.2.1 - 22.2.4 expires Jun 23, 2024
22.2.5 - 22.2.8 expires Dec 19, 2024
22.2.9 - 22.2.11 expires May 28, 2025
Powered by Wolken Software