Error: "ORA-01691: unable to extend lob segment"

Products

• Data Loss Prevention Enforce

Issue/Introduction

Symantec Data Loss Prevention (DLP) Enforce server is filling up with .bad files located in your Data Directory:

• ProgramData\Symantec\DataLossPrevention\ServerPlatformCommon\Incidents
• var/Symantec/DataLossPrevention/ServerPlatformCommon/Incidents

Additionally, a 404 browser redirect error may be present when navigating in the Enforce Server console.

IncidentPersister0.log

Caused by: com.vontu.incidenthandler.TransientPersistingException: com.vontu.model.DataAccessRuntimeException: org.apache.ojb.odmg.TransactionAbortedExceptionOJB: Can't commit objects:
* SQLException during execution of sql-statement:
* sql statement was 'INSERT INTO MessageLob (messageLobID,messageID,networkOriginalMessage,keyAlias) VALUES (?,?,?,?) '
* Exception message is [ORA-01691: unable to extend lob segment PROTECT.SYS_LOB00000216545649C00003$$ by 2768 in tablespace LOB_TABLESPACE
]
* Vendor error code [1691]
* SQL state code [72000]

IncidentPersister.log may also contain following entries as well.

Unable to extend Tablespace USERS
Unable to extend Tablespace LOB_TABLESPACE

 

Cause

This is due to the named TABLESPACE in the DLP Oracle database running out of space which then prevents the Enforce Server from processing incidents.

Resolution

Confirm whether either the LOB or USERS TABLESPACE is full:

1. Log into the Enforce console.
2. Go to  System > Database > Tablespaces Summary
3. Find the entries for LOB_TABLESPACE or USERS in the Tablespaces Allocation table.

NOTE: If you cannot access the console, you may run the following query in sqlplus as sysdba to see tablespace usage instead: 

-------------------------------------------------------------------------------

set pages 100
set lines 100
SELECT d.status "Status",
  d.tablespace_name "Name",
  TO_CHAR(NVL(a.bytes / 1024 / 1024, 0),'99999990D900') "Size (M)",
  TO_CHAR(NVL(NVL(f.bytes, 0), 0)/1024/1024 ,'99999990D900') "Free (MB)",
  TO_CHAR(NVL((NVL(f.bytes, 0)) / a.bytes * 100, 0), '990D00') "Free %"
 FROM sys.dba_tablespaces d,
  (select tablespace_name, sum(bytes) bytes from dba_data_files group by tablespace_name) a,
  (select tablespace_name, sum(bytes) bytes from dba_free_space group by tablespace_name) f
  WHERE d.tablespace_name = a.tablespace_name(+)
  AND d.tablespace_name = f.tablespace_name(+)
  AND NOT (d.extent_management like 'LOCAL'  AND d.contents like 'TEMPORARY')
 order by "Free %";

-------------------------------------------------------------------------------

 

You should see an output similar to the following with a lower FREE % nearing 0%:

 

To resolve the issue:

1. Increase the size of the LOB_TABLESPACE (see Oracle tablespace (LOB_TABLESPACE, USERS, etc.) for DLP is full, almost full, or critically full (broadcom.com)).
2. You have the following two options to clear up the .bad files
   1. Rename the .bad files  to .idc which should allow the incidents to be processed (Recommended)
   2. Delete the .bad files (These incidents will be permanently lost)

References:

What is a .bad file? (broadcom.com)

Where is the Oracle alert_protect.log? (broadcom.com)

Information about the LOB_TABLESPACE in DLP (broadcom.com)

Oracle tablespace (LOB_TABLESPACE, USERS, etc.) for DLP is full, almost full, or critically full (broadcom.com)