SpectroSERVER crashed after DX NetOps 20.2 upgrade
Article ID: 195212
Updated On:
Products
Issue/Introduction
We have upgraded our Spectrum 10.2.1 to the latest NetOps 20.2. (Spectrum 10.4.2). Our DSS servers upgraded and are running fine. However, our MLS main location server will crash a few seconds after startup. The SS will not activate.
There is no CORE file and the $SPECROOT/SS/VNM.OUT file shows heap corruption:
*** Error in `/app/spectrum/SS/SpectroSERVER': corrupted size vs. prev_size: 0x000000000bc14c60 ***
Cause
What appears to have happened is a permissions issue on the server caused by the upgrade. NetOps 20.2 allows Spectrum to be installed as a non-root user. However older versions such as 10.2.1 this was not the case. Thus, in the original 10.2.1 install, many files (especially those for use in processd) were owned by root user. When the upgrade-in-place was done by non-root user, it created some permissions conflicts. This was the root cause of the crashing.
Environment
Release : 10.4.2
Component : Spectrum Core / SpectroSERVER
Resolution
To solve the issue:
1. run the provided script /Install-Tools/CASpectrum_ProcessRegister.sh as root user to reset installation permissions
2. Initialize database to $SPECROOT/SS/legacy.SSdb catalog
cd $SPECROOT/SS
run ../SS-Tools/SSdbload -ic legacy.SSdb
3. load the backup <db_filename>.SSdb models
../SS-Tools/SSdbload -m <db_filename>.SSdb
4. Switch the SpectroSERVER binary to the mts version
cd $SPECROOT/SS
mv SpectroSERVER SpectroSERVER.1042.orig
cp -p support/SpectroSERVER_mts ./SpectroSERVER
Note: verify that the owner/group and permissions match the SpectroSERVER.1042.orig we renamed earlier and modify if needed
This crash is resolved on 10.4.2.0.189 with patch 10.04.02.PTF_10.4.213.
This will be resolved in DX NetOps Spectrum 20.2.1 and above.
Additional Information
On a new installation of NetOps 20.2, or a fresh installation (migration upgrade) this is not an issue.
If you are upgrading Spectrum from a version earlier than 10.4.0, recommend to run the install as root user:
Feedback
