Backup failed: No space left on device

Products

• STARTER PACK-7
• CA Rapid App Security
• CA API Gateway

Issue/Introduction

I am getting a 'No space left on device' error when executing the with ssg backup. I looked at the disk spaces and there appears to  be enough space. 

​INFO: Backing up component maindb 
Aug 16, 2018 12:42:50 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Dumping database to /tmp/ssg_backup_restore6966816463822216085tmp/maindb .. 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'maindb' could not be backed up: Cannot back up the database component: No space left on device 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component config 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'config' could not be backed up: Cannot back up ssg configuration: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/config 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component os 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'os' could not be backed up: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/os 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component ca 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'ca' could not be backed up: Cannot back up custom assertions: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/ca 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component ma 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'ma' could not be backed up: Cannot back up modular assertions: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/ma 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 

Cause

The database was very large due to an excessive amount of INSERT statements into the 'message_context_mapping_values' table. These entries need to be removed before completing the backup.

Environment

Release:
Component: APIESM

Resolution

1. Stop the ssg service on all nodes. 
2. Go to the mysql prompt of the primary node 
3. Run the following mysql command: 
    use ssg; 
4. Run the following mysql commands (you can bulk copy/paste into the mysql prompt): 
    SET FOREIGN_KEY_CHECKS = 0; 
    truncate table service_metrics_details; 
    optimize table service_metrics_details; 
    truncate table service_metrics; 
    optimize table service_metrics; 
    truncate table message_context_mapping_values; 
    SET FOREIGN_KEY_CHECKS = 1; 
5. Make sure you hit enter one more time so that the last command is executed. 
6. Exit mysql, and start the ssg service on all nodes.