Enabling request logging in knative serving to review access log in each Namespace

Products

• VMware Tanzu Application Platform

Issue/Introduction

Usually the user can use the command kubectl logs -l app=envoy -c envoy -n tanzu-system-ingress to review the HTTP request details/histories for the apps running on Tanzu Application Platform (TAP) and below HTTP histories shall return. 

[2024-09-24T13:20:26.800Z] "GET / HTTP/2" 200 - 0 91 6410 6409 "x.x.x.x" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "###" "tanzu-java-web-app.example.com" "y.y.y.y:8012"
[2024-09-24T13:20:33.492Z] "GET /favicon.ico HTTP/2" 404 - 0 119 58 57 "x.x.x.x" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" "25b66258-2708-4efa-b99b-cf93a87c7e4a" "tanzu-java-web-app.example.com" "y.y.y.y:8012"

However, in some scenarios, there is a need to check the HTTP requests in each developer Namespace where the app is running, instead of a unified Namespace (tanzu-system-ingress). 

 

Resolution

The solution in TAP currently is to enable the request logging setting in knative serving. Please reference https://knative.dev/docs/serving/observability/logging/request-logging/ to learn more about the request logging setting. 

Please reference below steps to enable this setting.

1. Enable request logging in knative serving.

a) kubectl get cm -n knative-serving config-observability -o yaml > config-observability.yml
b) and edit config-observability.yml

apiVersion: v1
data:
  logging.enable-request-log: "true"
  logging.request-log-template: '{"httpRequest": {"requestMethod": "{{.Request.Method}}", "requestUrl": "{{js .Request.RequestURI}}", "requestSize": "{{.Request.ContentLength}}", "status": {{.Response.Code}}, "responseSize": "{{.Response.Size}}", "userAgent": "{{js .Request.UserAgent}}", "remoteIp": "{{js .Request.RemoteAddr}}", "serverIp": "{{.Revision.PodIP}}", "referer": "{{js .Request.Referer}}", "latency": "{{.Response.Latency}}s", "protocol": "{{.Request.Proto}}"}, "traceId": "{{index .Request.Header "X-B3-Traceid"}}"}'
  _example: |
...

c) kubectl delete -f config-observability.yml && kubectl apply -f config-observability.yml

2. Once the configmap is updated, access logging should be enabled automatically.

kubectl logs -n <DEVELOPER NAMESPACE> service/<WORKLOAD NAME>-00001-private -c queue-proxy -f
kubectl logs -n <DEVELOPER NAMESPACE> deployment.apps/<WORKLOAD NAME>-00001-deployment -c queue-proxy -f
tanzu apps workload tail <WORKLOAD NAME> --namespace <DEVELOPER NAMESPACE>

 

Please contact the Tanzu support team and create a support ticket shall you have any questions regarding the above procedure.