Large number of 0 byte files in databases directories on linux client

Products

• Endpoint Security Complete
• Endpoint Protection

Issue/Introduction

While checking Symantec folder, there are 1000's of 0 byte log files under below paths.

/opt/Symantec/sdcssagent/AMD/sef/appdata/databases/*/*

These databases are commonly known as RocksDB or Write Ahead Logs.

Environment

Release : 14.3 RU5

Linux

Cause

This is what usually happens with a rocksdb database:
1. A rocksdb database is opened. Upon opening, a new 0-byte WAL (e.g. 000005.log) is created
2. Data is written to the databases and also to the WAL from process' memory.
3. The database performs a flush (often due to the request of the database's client or when the database is closed). The data in the WAL is persisted to a SST file (e.g. 000006.sst). The WAL file(s) is now obsolete and deleted.

However, if there is no data written to the database, the WAL (.log) will remain at 0-byte and flushing in this case wouldn't create a SST file and the WAL (.log) file will not get deleted. When the database is re-open, a new WAL (.log) with a higher sequence no. will be created.

In summary, every open-close of a rocksdb database without any data written will add a 0-byte .log file to the directory.

Resolution

Symantec is aware of this issue and a fix is underway. It will be included in a future build.

In the meantime it is safe to delete these files using these steps:

Stop the agent by running ./stop.sh from /usr/lib/symantec
Delete /opt/Symantec/sdcssagent/AMD/sef/appdata/databases/file_reputation/*                                             
Delete /opt/Symantec/sdcssagent/AMD/sef/appdata/databases/scheduler/*
Delete /opt/Symantec/sdcssagent/AMD/sef/appdata/databases/EDR/*      (Note: DO NOT delete this directory if the endpoint has EDR enabled)
Restart the agent by running  ./start.sh from /usr/lib/symantec