File Reader fails to start until server is rebooted

Products

• Data Loss Prevention Endpoint Prevent
• Data Loss Prevention Network Monitor
• Data Loss Prevention Network Prevent for Email
• Data Loss Prevention Network Discover
• Data Loss Prevention

Issue/Introduction

FileReader fails to start until Server is rebooted.
Restarting the services on Enforce or the server itself does not allow FileReader to start and FileReader will only start when the machine is rebooted.

Reviewing the Filereader0.log shows the following error:

SEVERE: [8676] Interprocess exception caught while opening server shared memory with error message - The system cannot find the file specified., Exception thrown from : ClientShmChannelImpl.cpp(80) HostManager.cpp 129
Feb 4, 2015 1:24:06 PM com.vontu.cracker.jni.NativeContentExtractionEngine create
SEVERE: [8676] Exception caught during starting up host manager. ContentExtractionEngineImpl.cpp 53
Feb 4, 2015 1:24:06 PM com.vontu.messaging.FileReaderSetup initialize
SEVERE: (DETECTION.3) Failed to initialize Detection
com.vontu.cracker.jni.NativeException: Failed to start Engine

Cause

FileReader fails to start if event id 6005 (EventLog Startup event) is missing from the windows system event queue.

If Windows Event Log doesn't contain the Startup Event (Event 6005) for any
reason, then get_bootstamp returns an empty string. CallStack:

shared_memory_object::priv_open_or_create
  ipcdetail::create_tmp_and_clean_old_and_get_filename
    create_tmp_and_clean_old
      tmp_folder
        get_bootstamp
          get_last_bootup_time

When create_tmp_and_clean_old tries to delete all the entries for the previous boot sessions, the empty string (that is delete all folders except "") causes it to delete the current session's files as well.

Workaround :

We can use PowerShell to emulate a reboot by introducing a windows reboot event.

1. Using Admin privileges on the detection server open up PowerShell.exe
2. Use the following PowerShell script to generate a windows event ID:

   • Write-EventLog –LogName System –Source “EventLog” –EntryType Information –EventID 6005 –Message “The Event log service was started.”

3. Restart the SymantecDLPServer service on the detection server.

 

 

 

Resolution

For each case we have seen with this appears to have Mcafee AV installed.

Check For Mcafee Antivirus and ensure proper exclusions are in place per Servers with Antivirus and Symantec Data Loss Prevention (DLP) Server Software (broadcom.com).

 

Additional Information

Other items which have been suggested for this issue:

• Check that mandatory directories and permissions are correct for the "SymantecDLP" user accounts (the service account, which used to be "protect" by default).
• That the user profile for the "SymantecDLP" service account have been created.
• Try setting a password for the "SymantecDLP" account and verify that you can login as those account. Make sure you update the password for the Vontu Monitor service if you do this.

In some cases some level of OS hardening has been performed via group policies or similar, this can prevent the profile for the "SymantecDLP" user from getting created correctly which in turn will cause our FileReader process to fail. The solution is to make the "SymantecDLP" user a member of the local Administrators group for the first startup of the SymantecDLPServer service. This will allow the profile to be created correctly, thereafter the "SymantecDLP" user can be removed from the local Administrators group.

On Linux based systems the following should be checked as a possible cause:

• Confirm that SELinux has been disabled.
• Confirm that /dev/shm is mounted via tmpfs in /etc/fstab not ramfs