Clarity Logs Show Error: No space left on device

Products

• Clarity PPM SaaS
• Clarity PPM On Premise

Issue/Introduction

One or more of the following symptoms occur:

• Clarity displays error: “System error. Contact system administrator.” when navigating to the Resource List View
• Jobs fail with error. Such as the Load Data Warehouse job, Rate Matrix Extraction job etc.

Example error stacks are as below.

In an Oracle DB based system:

java.io.IOException: No space left on device at java.base/java.io.UnixFileSystem.createFileExclusively(Native Method) at java.base/java.io.File.createNewFile(File.java:1024) at org.apache.lucene.store.SimpleFSLock.obtain(SimpleFSLockFactory.java:144) at org.apache.lucene.store.Lock.obtain(Lock.java:73) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:691) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:452) at com.niku.search.engine.lucene.LuceneIndexEngine.optimizeIndex(LuceneIndexEngine.java:694) at com.niku.search.engine.lucene.LuceneIndexEngine.optimizeSearchContentIndex(LuceneIndexEngine.java:678) at com.niku.search.engine.lucene.LuceneIndexEngine.index(LuceneIndexEngine.java:766) at com.niku.search.nspider.Nspider.scheduledEventFired(Nspider.java:187) at com.niku.njs.Dispatcher$BGTask.run(Dispatcher.java:657) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

In a PostgreSQL DB based system:

Refresh Security Materialized Views - An error occurred executing this job entry : 
Couldn't execute SQL: CALL DWH_SECURITY_VIEWS_REFRESH_SP()ERROR: 
could not write to file "pg_tblspc/7398075/PG_14_202107181/pgsql_tmp/pgsql_tmp17350.2088": No space left on device
Where: parallel workerSQL statement "CREATE MATERIALIZED VIEW dwh_sec_inv_inst_security_mv  

Environment

All Clarity versions

Cause

These error messages have been noticed in the log files (such as the app-ca.log and bg-ca.log):

Error message: [CA Clarity][Oracle JDBC Driver]Error writing data to client-side row cache.
 Verify that the directory specified by the java.io.tmpdir property exists and the current user has permission to read from and write to that directory. 
SQL error code: 0 

Error message: [CA Clarity][Oracle JDBC Driver]IO Error writing temp file: No space left on device

Resolution

The Infrastructure team will need to look at the DB connected to the application, and clear out log files from the drive supporting the Clarity server because it is full.