Problem
eHealth running on Windows hosts has a known issue whereby not all UDP based traps sent to their destinations are sent. Many are lost to the OS layer after being passed to the OS for send to the destination host. Most often this problem is observed by users having traps sent to CA Spectrum. It is most often observed by randomly missing alarms being raised or cleared in Spectrum from eHealth traps.
This can be observed in rare cases for Linux systems. Due to this a version of the tool is also available for Linux, with instructions for its use included below.
Question
How do you install and configure the eHealth AlarmForwarder solution?
Solution
In earlier CA Spectrum and CA eHealth releases, a solution was created utilizing two TrapEXPLODER installations, one on the eHealth server receiving traps from eHealth and sending them on to Spectrum. One on the Spectrum server receiving traps from the eHealth host and processing them to the Spectrum application on the destination server. This was a somewhat difficult solution to implement. Due to that a new tool was provided beginning with eHealth release 6.3.2.07.
That new tool is called the AlarmForwarder. It consists of two parts:
Part 1: The AlarmForwarder.sh script that runs on eHealth:
Part 2: The GdEventServlet that runs on the Spectrum OneClick tomcat web server:
Installation (two parts)
eHealth Installation (Part 1):
**Repeat these steps on all eHealth servers polling devices and generating Live Health Alarms**
1. The alarmForwarder.tar.gz is found in the $NH_HOME/modules directory. Copy it to the $NH_HOME/custom directory.
2. Extract the tool to its own new directory $NH_HOME/custom/alarmForwarder with the following command:
tar -xvf alarmForwarder_<platform>.tar $NH_HOME/custom/alarmForwarder
3. Update the $NH_HOME/custom/alarmForwarder/alarmForwarderIni.pm file. Set the correct values for this environment as follows:
our $spectrumProtocol = 'http'; (set this to the protocl Spectrum uses, http or https)
our $spectrumServer = 'yourSpectrumOCEServer'; (set the resolvable Spectrum OCE host name here)
our $spectrumPort = '8080'; (set the correct port the Spectrum OCE web server works on)
our $spectrumUser = 'spectrum'; (set the spectrum default admin user here)
our $spectrumPass = 'spectrum'; (set the spectrum default admin users password here)
our $eHealthServer = 'yourEHealthServer'; (set the model name that represents the eHealth sever in Spectrum here)
our $eHealthServerIP = 'yourEHealthServersIP'; (set the eHealth servers IP address here)
our $eHealthServerPort = '80'; (set the port the eHealth web server runs on here)
our $EventAdminMH = 'yourEventAdminModel_Handle'; (0x123456 -> Need to model all eHealth BE’s in one SpecServer and get the ModelHandle)
4A. Windows OS specific details:
- Specify the variable TZ (time zone) in the file alarmForwarder.bat, (e.g. “set TZ=EST5EDT”)
- Copy the batch file alarmForwarder.bat from $NH_HOME/custom/alarmForwarder to $NH_HOME/bin/sys/
- Add the alarmForwarder.bat to the ehealth startup.cfg file so it starts/stops with the eHealth server service(s):
Edit $NH_HOME/sys/startup.cfg and add the following (place before the ‘}’):
program alarmForwarder.bat {
restart this # Restart only this server if it fails
wait 15
}
- Restart eHealth server services via the Services UI, or at the command line with the commands:
nhServer stop
nhServer start
- NOTE: This solution uses the environment variable %NH_USER% to determine the Oracle TNS connection string. It has been observed that this does not work on some Windows servers. To test, try connecting to the local eHealth database instance using the following command line:
SqlPlus ehealth/[email protected]_EHEALTH
Where ehealth/ehealth is default ehealth install owner admin user
Where you also must add '@HOSTNAME_DBNAME'
If any of the necessary values are unknown, to figure out what the are:
The $NH_USER command, if unknown, can be found running this command:
ENV | grep -i NH_USER
The Oracle DB name, if unknown, is in the ENV variable ORACLE_SID. To find it run this in the command prompt:
ENV | grep -i ORACLE_SID
As an additional alternative, if you run this command:
ENV | grep -i CONNECT_STRING
You should see an ENV variable named NH_DB_CONNECT_STRING. Its value is what you would place after the '@' symbol in the command. For example if my user is 'eHealth', my host name is 'EHSVR1' and my DB name is 'DATA' the command would be:
SqlPlus eHealth/[email protected]_DATA
The resulting output show show something similar to this:
SQL*Plus: Release 10.2.0.3.0 - Production on Fri Mar 7 11:32:57 2014
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL>
If you receive the message, “ERROR: ORA-12560: TNS:protocol adapter error”, then a new variable, %NH_DBUSER%, must be configured. The %NH_DBUSER% will be:
%NH_USER%@<hostname>_<ORACLE_SID>
Where %NH_USER% is the eHealth user account for this system, <hostname> is the hostname of the server and <ORACLE_SID> is the database name.
AlarmForwarder_DBUSER_Variable.png
4B. Linux OS specific details: There are only three small differences configuring the tool in Linux.
- Linux systems have not been seen to need the %NH_DBUSER% environment variable added
- Linux systems use the alarmForwarder.sh (Windows uses the .bat version). Ensure to note the difference for the startup.cfg file edits.
- eHealth server stop/start is only available via the command line.
Spectrum Installation (Part 2):
**Repeat these steps on all Spectrum OneClick tomcat web servers in the environment that are targeted to receive these Live Health alarms**
1. To install the required Spectrum files begin by copying the restevent.tar.gz from the files extracted into the $NH_HOME/custom/alarmForwader directory on the eHealth host. Copy it over to the $SPECROOT directory on the target Spectrum hosts.
2. Expand the file to extract its contents. When extracted properly we should see the following files show up on the Spectrum OCE server:
$SPECROOT\tomcat\lib\restevents.jar
$SPECROOT\tomcat\webapps\spectrum\WEB-INF\restful\schemas\Event.xsd
$SPECROOT\tomcat\webapps\spectrum\WEB-INF\restful\schemas\Request.xsd
Note: Before running the tar extraction steps below, it is recommend that a back up copy of the files in those folders be made as a precaution. The backup copy can be deleted after extfaction occurs and everything runs properly.
First extract the GZ file content using this command:
gunzip restevent.tar.gz
Now extract the compressed tar file with the command:
tar -xvf restevent.tar
3 - Enable the GdEventServlet to run in the web server. This is done by editing the $SPECROOT/tomcat/webapps/spectrum/WEB-INF/web.xml file. Please pay special attention when editing this file. Try to edit it with WordPad or other simplistic text editor for best results. The key here is to ensure not to use tabs in this file, only spaces. Using tabs breaks the web server and this will never work properly.
Start by makeing a back up copy of the web.xml file should it be needed. Next edit the $SPECROOT/tomcat/webapps/spectrum/WEB-INF/web.xml and find this section:
<init-param>
<param-name>jaxrs.serviceClasses</param-name>
<param-value>
com.ca.spectrum.restful.servlet.ActionServlet
com.ca.spectrum.restful.servlet.AlarmServlet
com.ca.spectrum.restful.servlet.AssociationServlet
com.ca.spectrum.restful.servlet.DeviceServlet
com.ca.spectrum.restful.servlet.LandscapeServlet
com.ca.spectrum.restful.servlet.ModelServlet
com.ca.spectrum.restful.servlet.ModelsServlet
com.ca.spectrum.restful.servlet.SubscriptionServlet
com.ca.spectrum.restful.servlet.AttributeServlet
com.ca.spectrum.restful.servlet.TestNotificationServlet
com.ca.spectrum.restful.servlet.ConnectivityServlet
com.ca.spectrum.restful.servlet.EventServlet
</param-value>
</init-param>
After the line with EventServlet add the line:
com.ca.spectrum.restful.servlet.GdEventServlet
The lines should then appear as follows:
<init-param>
<param-name>jaxrs.serviceClasses</param-name>
<param-value>
com.ca.spectrum.restful.servlet.ActionServlet
com.ca.spectrum.restful.servlet.AlarmServlet
com.ca.spectrum.restful.servlet.AssociationServlet
com.ca.spectrum.restful.servlet.DeviceServlet
com.ca.spectrum.restful.servlet.LandscapeServlet
com.ca.spectrum.restful.servlet.ModelServlet
com.ca.spectrum.restful.servlet.ModelsServlet
com.ca.spectrum.restful.servlet.SubscriptionServlet
com.ca.spectrum.restful.servlet.AttributeServlet
com.ca.spectrum.restful.servlet.TestNotificationServlet
com.ca.spectrum.restful.servlet.ConnectivityServlet
com.ca.spectrum.restful.servlet.EventServlet
com.ca.spectrum.restful.servlet.GdEventServlet
</param-value>
</init-param>
4. Save the changes to the web.xml file and restart the Spectrum Tomcat OneClick web server services to have it try to start the new GdEventServlet. A simple method of doing so is to run the command:
$SPECROOT/tomcat/webapps/spectrum/restart.bat
5. Next, On the MLS, we'll need to edit the $SPECROOT/SS/CsVendor/gen_app_gw/EventAdmin/EventDisp by adding the following lines:
0x00011002
0x00011003
5a. Reload the event configuration files.
* To update the overall alert and event system more broadly for a given landscape, in OneClick, click the Update Event Configuration button that is available on the SpectroSERVER Control subview of the Information tab on the VNM model.
OR
* Update action=0x100a2 mh=<VNM modelhandle>
Important! This update process flushes and reloads all event rules including those that are in the middle of processing. As a consequence, the processing of events by active event rules is aborted, and all associated data (for example, counts for occurrences of contributing events) is lost.
If no errors are observed related to the GdEventServlet startup or operation in the Spectrum web server logs the tool is functioning. It is the same for the alarm-forwarder.log file that should be written to the $NH_HOME/custom/alarmForwarder.
Release: SPDBIP99000-9.4-Spectrum-Device Based Suite-IP Services Manager
Component:
Please Update This Required Field