How to create a custom Spectrum ParseMap file
Article ID: 52128
Updated On: 10-27-2023
Products
Issue/Introduction
Environment
Component:
Resolution
When using an agent to do log file scraping, a Spectrum ParseMap file can be used to enhance the processing of the event created. ParseMap files specify the event associated with the information in the incoming logfile trap. In addition, ParseMap files allow you to specify that portions of the log file entry text be used as event variables. If desired, these variables can be used in conjunction with an Event Rule to process the event.
Log File Syntax
You can monitor application logs or log files that receive data from other devices, such as Syslog files. No special syntax is required for log files that monitor application logs. However, for SPECTRUM to assert the trap information on the appropriate device model and for ParseMap processing, log files that receive information from devices on the network must have the following format, which is generally based on the BSD Syslog and Cisco IOS format:
<MessagePrefix>%<MessageHeader><Additional_Information>
Where:
<MessagePrefix> contains the date and time of the message and the IP address or the host name of the source of the information contained in the entry. There can be other information interspersed within the prefix, but it must contain these two pieces of information.
Note: If a host name is used to identify the source, it can be of the form myhost.ca.com or myhost.
<MessageHeader> must have the format <A>-<B>-<C>
Where:
<A> contains any number of uppercase alpha characters, underscores, or the string "Aprisma."
<B> contains any number of uppercase alpha characters, numeric characters, or underscores.
<C> contains any number of uppercase alpha characters, underscores, or dashes.
<Additional_Information> can contain any data.
In general, this syntax can be found in the following types of log files:
Solaris syslog file entries from a Cisco or Riverstone device.
Solaris syslog file entries from another type of device that uses the <MessageHeader> format described in previously.
Kiwi syslog file entries from a Cisco or Riverstone device.
Kiwi syslog file entries from another type of device that uses the <MessageHeader> format described previously.
CA log files, for example those generated by SPECTRUM Report Gateway.
.
As described in Log File Syntax previously, a log file entry contains the following components:
<MessagePrefix>%<MessageHeader><Additional_Information>
SPECTRUM identifies the ParseMap file that will process the trap by finding the ParseMap file whose name matches the text of the <MessageHeader> from the log file entry. The following is an example of a log file entry:
27 Jul 14:30:01 <IP_Address> %NTP-3-STAT, server <IP_Address>, stratum 1, offset 0.000054, delay 0.02673
The <MessageHeader> portion of the entry is "NTP-3-STAT", so SPECTRUM looks for a ParseMap file named NTP-3-STAT. As such, you must create a ParseMap file for each log entry with a unique <MessageHeader> that you configure to generate a trap.
Note: There are many ParseMap files available for use in SPECTRUM. You can find them in the following directory: <$SPECROOT>/SS/CsVendor/ParseMaps.
To create a ParseMap file
1. Create a new text file using any text editor in the $SPECROOT/SS/CsVendor/ParseMaps directory
2. In the first line of the text file, type the new event file name for the event that you want to generate. The event file name must begin with "Event" and end with "xxxxxxxx" where x is be any valid hexadecimal number.
For example, Eventffff1a2f and Eventffff1234 are valid event file names. Event012za8b is not.
3. Create a new line in the text file by pressing the Enter key on your keyboard.
4. This line contains the <Additional_Information> portion of the log file entry. You can specify portions of this text as event variables, which can be used to process the event with an Event Condition rule.
The following is the ParseMap file for the NTP-3-STAT log file example listed above:
Eventffffffff
server {IPADDRESS 1}, stratum {INTEGER 2}, offset {STRING 3}, delay {STRING 4}
In the above ParseMap file, the words "server", "stratum", "offset", and "delay" along with the spaces and the commas "," will be consistent in ALL NTP-3-STAT log file entries. All constant text, spaces and special characters (like the commas in the NTP-3-STAT example) must be accounted for in the ParseMap file. Failure to account for these will result in a parsing error processing the log file.
The variables {IPADDRESS 1}, {INTEGER 2}, {STRING 3}, and {STRING 4} could be different for each individual NTP-3-STAT log file entry. These values in the associated NTP-3-STAT log file will be stored as variables (VARBIND) for the resulting event and can be used to enhance the processing of the event with an Event Condition rule.
Eventffffffff is the event that will be associated with this ParseMap file. An associated Event Format file will be needed for this event. The following is an example of the Event Format file for the Eventffffffff event:
{d "%w- %d %m-, %Y - %T"} - {d "%w- %d %m-, %Y - %T"} - %NTP-3-STAT, server {S 1}, stratum {I 2}, offset {S 3}, delay {S 4} (event [{e}])
This Eventffffffff event is required to be defined in an EventDisp file for logging and alerting purposes. The following is an example of an EventDisp file entry for the Eventffffffff event:
0xffffffff E 50
5. Save the text file in the <$SPECROOT>/SS/CsVendor/ParseMaps directory. The name of this text file must match the <MessageHeader> portion of the log file entry. In this case the file name would be "NTP-3-STAT". Do not include a file extension in the file name.
Note: Only the first two lines of the ParseMap file are processed. You can put whatever you want on subsequent lines but they are not processed and are informational only.
.
Valid STRING Data Types
The following are valid data types for use in variables.
STRING Matches all string characters up to the next literal, data type or end of string.
STRINGNOWS Matches all string characters up to the next space, literal, data type, or end of string.
INTEGER Matches any positive integer value.
IPADDRESS Matches any valid IPv4 address.
Whitespace in STRING Variables
Since whitespace is a valid character in the definition of the STRING variable, you should always separate multiple STRING tokens with recognizable patterns.
For example, a valid ParseMap entry could contain entries that look like these:
{STRING 1}, {STRING 2}
{STRING 1}
{IPADDRESS 2} {STRING 3}
{STRING 1} literal text {STRING 2}
{STRINGNOWS 1} {STRING 2}
However, you should not have these because the resulting regular expression will be ambiguous and will fail to parse properly:
{STRING 1}{STRING 2}
{STRING 2} {STRING 2}
Additional Information
Feedback
