crictl logs <container_id> reports “wal.broken: no space left on device”

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

crictl logs <container_id> reports “wal.broken: no space left on device”

Running 'df -h' shows root partition at 100% full

Large file (10+ GB): /var/log/messages

 

Environment

VKS 1.33

Cause

Root partition is full.

Resolution

Truncate the messages file to 0 bytes (do not delete the file itself):
truncate -s 0 /var/log/messages

Remove any older compressed archives that may be taking up space:
rm -rf /var/log/messages.*.gz