Enable Audit Logging in VMware vSphere Kubernetes Service(VKS) Environments
search cancel

Enable Audit Logging in VMware vSphere Kubernetes Service(VKS) Environments

book

Article ID: 425040

calendar_today

Updated On:

Products

Tanzu Kubernetes Runtime VMware vSphere Kubernetes Service

Issue/Introduction

Customers running vSphere Kubernetes Service(VKS) environments may need to verify, customize, or export Kubernetes API server audit logs to meet organizational compliance, security auditing, or operational troubleshooting requirements.

Note: Kubernetes API audit logging is enabled by default in modern vSphere Kubernetes Service(VKS) clusters. Manual enablement from a disabled state is generally not required.

The purpose of this article is to provide updated technical guidance on verifying active audit logging, customizing audit policies, and managing log retention/forwarding for guest clusters.

Environment

vSphere Kubernetes Service(VKS) 

Cause

The Kubernetes API server requires specific startup flags and a policy configuration file to capture and persist API requests.

In vSphere Kubernetes Service(VKS) guest clusters, the deployment automation pre-configures the kube-apiserver static pod with the required audit flags upon cluster creation.

Resolution

Default Configurations:

  • VKS Workload (Guest) Clusters:

    • Log File Location: /var/log/kubernetes/kube-apiserver.log

    • Audit Policy Location: /etc/kubernetes/extra-config/audit-policy.yaml

    • Default API Server Flags:

      - --audit-log-maxage=30
      - --audit-log-maxbackup=10
      - --audit-log-maxsize=100
      - --audit-log-path=/var/log/kubernetes/kube-apiserver.log
      - --audit-policy-file=/etc/kubernetes/extra-config/audit-policy.yaml
      
  • Supervisor Clusters:

    • Log File Location: /var/log/vmware/audit/kube-apiserver.log

    • Audit Policy Location: /etc/vmware/wcp/audit-policy.yaml

Resolution

Step 1: Verify Audit Logging Status on a Cluster

To verify that API audit logging is active on a Control Plane node:

  1. Connect to the Control Plane node via SSH.

  2. Verify that the kube-apiserver process is running with audit flags:

    ps -ef | grep kube-apiserver | grep -E "audit-log-path|audit-policy-file"
    
  3. Check that the log file exists and is recording live audit events:

    tail -f /var/log/kubernetes/kube-apiserver.log
    

Step 2: Customize the Audit Policy (Optional)

By default, the API server logs events according to the baseline rules defined in the cluster's audit policy file. If your compliance team requires modified verbosity (e.g., logging full RequestResponse payloads for specific resources):

  1. Open the audit policy file on the Control Plane node:

    sudo vi /etc/kubernetes/extra-config/audit-policy.yaml
    
  2. Modify the rules using standard Kubernetes audit policy syntax (e.g., set level: RequestResponse or level: Metadata for targeted API groups).

  3. Save the file.

  4. Restart the API server pod to ensure changes are applied immediately:

    sudo crictl stop $(sudo crictl pods --name kube-apiserver -q)
    

    (The kubelet will automatically restart the kube-apiserver container within a few seconds).

 

Verification Example

To extract and format a single audit log event to confirm field coverage:

grep -i '"verb":"create"' /var/log/kubernetes/kube-apiserver.log | head -n 1 | jq '{verb: .verb, user: .user.username, objectRef: {resource: .objectRef.resource, name: .objectRef.name, namespace: .objectRef.namespace}, sourceIPs: .sourceIPs, userAgent: .userAgent, requestReceivedTimestamp: .requestReceivedTimestamp}'

Additional Information

Kubernetes Documentation: Auditing

Workload Security & Audit Logging