Customer face the issue "One or more resources might be missing because of cluster query errors." on TAP-GUI with TAP multi-cluster

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Customer using TAP multi-cluster will face below error in Runtime Resources of TAP-GUI.

Click "Show details", you will see error messages like below:

Access error when querying cluster '<cluster_name>' for resource '/apis/conventions.carto.run/v1alpha1/podintents' (status: 403). Contact your administrator.
Knative is not installed on '<cluster_name>' (status: 404). Contact your administrator.
Error when querying cluster '<cluster_name>' for resource '/apis/tekton.dev/v1beta1/pipelineruns' (status: 404). Contact your administrator.

Environment

Product Version: 1.3

Resolution

In our doc Errors retrieving resources, there are some common troubleshooting steps for errors presented in the Runtime Resources tab.

For error message "Access error when querying cluster '<cluster_name>' for resource '/apis/conventions.carto.run/v1alpha1/podintents' (status: 403). Contact your administrator.", it indicates that ClusterRole don't have the privilege 'get', 'watch', 'list' on resource 'podintents' of apiGroups 'conventions.carto.run' on the querying cluster. To workaround this, add below code snippet in the tap-gui-viewer-service-account-rbac.yaml and apply yaml file on the querying cluster.

- apiGroups: ['conventions.carto.run']
  resources:
  - podintents
  verbs: ['get', 'watch', 'list']

All of these 404s would be expected. It has to do with the fact that the backstage kubernetes backend queried the same set of CRDs for every clusters. So your backstage instance is trying to look up resource or package on the cluster which is not supposed to be installed. These 404s won't actually create an issue in the use of RRV or the Supply chains plugin etc. You can use below workaround to make them go away or wait a fix in future release. This known issue can be traced from upstream https://github.com/backstage/backstage/issues/15994. It will be fixed in Backstage v1.16.0 and should be shipped with TAP 1.7. 

For the error message "Knative is not installed on '<cluster_name>' (status: 404). Contact your administrator.", it indicates that the cluster does not have Cloud Native Runtimes installed.
To workaround this, install the Knative components by following the instructions in Install Cloud Native Runtimes.
Step 1. Create a cnr-values.yaml by using the following sample as a guide:

---
domain_name: <domain_name>
ingress:
  external:
    namespace: tanzu-system-ingress
  internal:
    namespace: tanzu-system-ingress

Step 2. Install the package by running:

$ tanzu package available list cnrs.tanzu.vmware.com --namespace tap-install
  NAME                   VERSION  RELEASED-AT                    
  cnrs.tanzu.vmware.com  2.0.2    2022-10-12 00:00:00 +0000 UTC 

$ tanzu package install cnrs -p cnrs.tanzu.vmware.com -v 2.0.2 -n tap-install -f cnr-values.yaml --poll-timeout 30m
 Updating installed package 'cnrs'
 Getting package install for 'cnrs'
 Getting package metadata for 'cnrs.tanzu.vmware.com'
 Updating secret 'cnrs-tap-install-values'
 Updating package install for 'cnrs'
 Waiting for 'PackageInstall' reconciliation for 'cnrs'
 'PackageInstall' resource install status: ReconcileSucceeded
 Updated installed package 'cnrs' in namespace 'tap-install'

For the error message "Error when querying cluster '<cluster_name>' for resource '/apis/tekton.dev/v1beta1/pipelineruns' (status: 404). Contact your administrator.", it indicates that the package that contains the resource is not installed. 
To workaround this, install the missing package on the querying cluster.

tanzu package available list -n tap-install tekton.tanzu.vmware.com
  NAME                     VERSION       RELEASED-AT                    
  tekton.tanzu.vmware.com  0.39.0+tap.2  2022-08-31 15:28:05 +0000 UTC 

tanzu package install tekton-pipelines -n tap-install -p tekton.tanzu.vmware.com -v 0.39.0+tap.2
 Installing package 'tekton.tanzu.vmware.com'
 Getting package metadata for 'tekton.tanzu.vmware.com'
 Creating service account 'tekton-pipelines-tap-install-sa'
 Creating cluster admin role 'tekton-pipelines-tap-install-cluster-role'
 Creating cluster role binding 'tekton-pipelines-tap-install-cluster-rolebinding'
 Creating package resource
 Waiting for 'PackageInstall' reconciliation for 'tekton-pipelines'
 'PackageInstall' resource install status: Reconciling
 'PackageInstall' resource install status: ReconcileSucceeded
 Added installed package 'tekton-pipelines'

Reference Links:
https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-tap-gui-cluster-view-setup.html
https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-tekton-install-tekton.html
https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-cloud-native-runtimes-install-cnrt.html
https://docs.vmware.com/en/VMware-Tanzu-Application-Platform/1.3/tap/GUID-tanzu-build-service-install-tbs.html