Backup failed: No space left on device

Products

• STARTER PACK-7
• CA Rapid App Security
• CA API Gateway

Issue/Introduction

I am getting a 'No space left on device' error when executing the with ssg backup. I looked at the disk spaces and there appears to  be enough space. 

​INFO: Backing up component maindb 
Aug 16, 2018 12:42:50 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Dumping database to /tmp/ssg_backup_restore6966816463822216085tmp/maindb .. 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'maindb' could not be backed up: Cannot back up the database component: No space left on device 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component config 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'config' could not be backed up: Cannot back up ssg configuration: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/config 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component os 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'os' could not be backed up: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/os 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component ca 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'ca' could not be backed up: Cannot back up custom assertions: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/ca 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
INFO: Backing up component ma 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 
WARNING: Component 'ma' could not be backed up: Cannot back up modular assertions: Unable to create directory: /tmp/ssg_backup_restore6966816463822216085tmp/ma 
Aug 16, 2018 12:44:09 PM com.l7tech.gateway.config.backuprestore.ImportExportUtilities logAndPrintMessage 

Environment

Release:
Component: APIESM

Cause

The database was very large due to an excessive amount of INSERT statements into the 'message_context_mapping_values' table. These entries need to be removed before completing the backup.

Resolution

1. Stop the ssg service on all nodes. 
2. Go to the mysql prompt of the primary node 
3. Run the following mysql command: 
    use ssg; 
4. Run the following mysql commands (you can bulk copy/paste into the mysql prompt): 
    SET FOREIGN_KEY_CHECKS = 0; 
    truncate table service_metrics_details; 
    optimize table service_metrics_details; 
    truncate table service_metrics; 
    optimize table service_metrics; 
    truncate table message_context_mapping_values; 
    SET FOREIGN_KEY_CHECKS = 1; 
5. Make sure you hit enter one more time so that the last command is executed. 
6. Exit mysql, and start the ssg service on all nodes.