'kapp list -A' shows Lcs column is false for one ore more rows but resource doesn't show any issue

Products

• VMware Tanzu Platform

Issue/Introduction

When listing all kapp applications on a cluster, 'Lcs' column (Last Change Successful) can show "false" but when checking the deployment, it was deployed successfully. 

E.g

kapp list -A
Target cluster 'https://xxx.xxx.xxx.xxx:6443' (nodes: workload-slot1rp4-controlplane-xrttr-rb679, 5+)

Apps in all namespaces

Namespace                     Name                                        Namespaces                          Lcs    Lca
default                       workload-slot1rp4-kapp-controller.app       (cluster),kube-system,tkg-system    true   10d
tkg-system                    cert-manager.app                            (cluster),cert-manager,kube-system  true   5d
^                             workload-slot1rp2-secretgen-controller.app  (cluster),secretgen-controller      false  10d
.
.
.
Lcs: Last Change Successful
Lca: Last Change Age

In the above example, workload-slot1rp2-secretgen-controller.app shows Lcs column as false, but when checking the app with kubectl, it shows it reconciled successfully.

$ kubectl get app workload-slot1rp2-secretgen-controller -n tkg-system
NAME                                    DESCRIPTION           SINCE-DEPLOY  AGE
workload-slot1rp2-secretgen-controller  Reconcile succeeded   3m23s         7h50m

Cause

This is a known issue, https://github.com/carvel-dev/kapp/issues/467 . This happens because Lcs is set to false(due to a timeout) before the deployment finishes, eventually resources succeeds reconciling but kapp doesn't get synced. 

Resolution

Currently, there is no plan to change this behaviour but making a change in the deployment to trigger reconciliation again will sync Lcs column. For that, follow steps below.

1. Annotate the deployment as follows "kubectl annotate deployment <deployment-name> -n <namespace> kapp.reconcile=true" E.g
   

   kubectl annotate deployment.apps/secretgen-controller -n secretgen-controller kapp.reconcile=true

2. Edit the app with "kubectl edit app <app-name> -n <namespace>" E.g
   

   kubectl edit app workload-slot1rp2-secretgen-controller -n tkg-system

3. Add "paused: true" below "spec:". E.g.

   spec:
     paused: true
     deploy:
     - kapp:
         rawOptions:

The above will trigger reconciliation and when running "kapp list -A" again, it will show Lcs value as true.