You have questions about: SSIM archives, raw events specifically syslog raw events
1.1 What is SSIM Event Archive? (All SSIM events)
- SSIM Archives are Symantec Security Information Manager’s proprietary, flat files based database. It is optimized for Indexing and Storing events at high speed as required by SSIM. SSIM Archives are designed to reduce Archive management cost and provide high flexibility in storing and maintaining event storage for longer duration.
- SSIM creates separate XFS partition for storing archives. This partition is created at /eventarchive folder in SSIM Server.
- SSIM event archiving is optimized for writing and retrieving events. All events are stored in flat files.
- Using well-structured file structure saves cumbersome database management for large number of the events
- Archive data is signed and compressed.
- Archives are signed and hashed.
- Archives are compressed using zlib library. The compression ratio achieved is 20%.
- Archive files are hashed to ensure that archives are not tampered. The default hashing algorithm used is MD5. It can be changed to SH1 or other supported algorithm by modifying /opt/Symantec/simserver/etc/event-service-startup.xml file.
- The Raw events are stored into SSIM Event Archive.
- Events are stored in name/value pairs which include a name (raw_events) for the field.
- Additional information is stored in fields as allows in the archive.
- Changes to the raw event field were made as an enhancement to the collector framework code for syslog events. This enhancement appended the Event Map to the original raw event to ensure context of the sending device would be retained. As tagged by event map designation, this contains critical context field values like sensor name, proxy IP/name, event date, time-offset which is used during translation rules & SES-Processor rules.
- The SSIM raw event field provides the original raw event received by the point product, as well as the original service map provided by the sensor, allowing for easy traceability of event lifecycle and additional troubleshooting.
2.1 Example of the raw data:
Below are sample syslogs with pre and post enhancement:
Before Live Update:
Raw Event = Jan 29 10:32:37 xxx.xxx.xxx.xxx 1689886: Jan 29 16:02:35.939 IST: %SEC-6-IPACCESSLOGP: list aclin denied tcp xxx.xxx.xxx.xxx(36303) -> xxx.xxx.xxx.xxx(49156), 1 packet
After Live Update:
Raw Event = Feb 06 21:20:42 172.31.98.80 Jan 29 10:32:37 xxx.xxx.xxx.xxx1689886: Jan 29 16:02:35.939 IST: %SEC-6-IPACCESSLOGP: list aclin denied tcp xxx.xxx.xxx.xxx(36303) -> xxx.xxx.xxx.xxx(49156), 1 packet (service map: <eventmap version="2"><field name="TimeOffset">0</field><field name="event_dt">1423237842545</field><field name="reporting_sensor">Sensor 0</field><field name="proxy_machine_ip">xxx.xxx.xxx.xxx</field><field name="proxy_machine">xxx.xxx.xxx.xxx</field></eventmap>)
As seen in this, the service map is appended to the raw event and distinguished with clear parentheses separators. The original raw event remains untouched.
3. What part of the raw data is filtered out to be stored?
- There are no filter applied on the raw events and they are stored “as is” in the Event Archive along with service map.
3.1 Why do we filter out certain fields?
- Not Applicable
4. How can the system reconstruct the raw data with the information available in the system?
- To view Raw Events stored into the Event Archive, Java based Thick Client is used.
- These events further can be exported into .csv format for investigation.
- This only applies if the collector has been enabled to retain the raw events.
5. How do we ensure events integrity? (What should we say to our customer if he ask about raw events integrity)
- The raw events received from the device are not tampered or filtered out since they are stored “as is” along with service map and additional headers appended.
- The event archive data is signed, compressed and the archives are signed and hashed.