Audit Log File /var/log/vmware/audit/kube-apiserver.log Remains Empty After Manual Recreation in vSphere Supervisor Cluster

Products

• Tanzu Kubernetes Runtime

Issue/Introduction

In vSphere Supervisor clusters, the audit log file located at /var/log/vmware/audit/kube-apiserver.log might be manually deleted during log cleanup. If the file is recreated using touch, it may appear present but remains empty, and audit logging does not resume.

This issue can occur when attempting to recover the audit log file without restarting the kube-apiserver process.

Cause

kube-apiserver uses the --audit-log-path flag to define the path where audit events are logged. The file is opened and its file descriptor is bound during kube-apiserver startup.

If the log file is manually deleted and recreated (e.g., using touch), kube-apiserver does not detect or rebind to the new file. As a result, no audit events are written, and the file remains empty.

Resolution

To restore audit logging functionality:

1. SSH into the affected supervisor control plane node.
2. Delete the manually created file if it exists:
   
   • Path: /var/log/vmware/audit/kube-apiserver.log
3. Identify the running kube-apiserver container:
   
   • Use crictl ps to find the container ID.
   • crictl ps | grep kube-apiserver
4. Restart the kube-apiserver container:
   
   • Run crictl stop <container_id>
5. kube-apiserver will restart automatically. Upon restart:
   
   • It will recreate the audit log file at the defined path.
6. Audit logging will resume as expected.