Multiple Instances of Package installation on the same cluster fails

Products

• Tanzu Kubernetes Grid

Issue/Introduction

Two instances of any User managed packages on the same cluster are not supported, this is applicable for TKGm (till version 2.3) and vSphere with Tanzu (till tanzu cli version 2.3) 

Resolution

Currently, there is no solution, you will see the below error in the PackageInstall CR 

kubectl -n tkg-system get pkgi contour2 -o yaml
apiVersion: packaging.carvel.dev/v1alpha1
kind: PackageInstall
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"packaging.carvel.dev/v1alpha1","kind":"PackageInstall","metadata":{"annotations":{},"name":"contour2","namespace":"tkg-system"},"spec":{"packageRef":{"refName":"contour.tanzu.vmware.com","versionSelection":{"constraints":"1.20.2+vmware.2-tkg.1"}},"serviceAccountName":"contour2-sa","values":[{"secretRef":{"name":"contour2-data-values"}}]}}
  creationTimestamp: "2023-08-28T04:43:53Z"
  finalizers:
  - finalizers.packageinstall.packaging.carvel.dev/delete
  generation: 1
  name: contour2
  namespace: tkg-system
  resourceVersion: "68159"
  uid: 1427c02c-ec2e-42dd-a20c-271ccceb098b
spec:
  packageRef:
    refName: contour.tanzu.vmware.com
    versionSelection:
      constraints: 1.20.2+vmware.2-tkg.1
  serviceAccountName: contour2-sa
  values:
  - secretRef:
      name: contour2-data-values
status:
  conditions:
  - message: Error (see .status.usefulErrorMessage for details)
    status: "True"
    type: ReconcileFailed
  friendlyDescription: 'Reconcile failed: Error (see .status.usefulErrorMessage for
    details)'
  observedGeneration: 1
  usefulErrorMessage: |-
    I0828 06:16:30.511411    1343 request.go:665] Waited for 1.038071397s due to client-side throttling, not priority and fairness, request: GET:https://10.245.0.1:443/apis/scheduling.k8s.io/v1?timeout=32s
    kapp: Error: Ownership errors:
    - Resource 'clusterrole/envoy (rbac.authorization.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'clusterrolebinding/envoy (rbac.authorization.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'clusterrole/contour (rbac.authorization.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'customresourcedefinition/contourconfigurations.projectcontour.io (apiextensions.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'clusterrolebinding/contour (rbac.authorization.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'customresourcedefinition/tlscertificatedelegations.projectcontour.io (apiextensions.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'customresourcedefinition/contourdeployments.projectcontour.io (apiextensions.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'customresourcedefinition/extensionservices.projectcontour.io (apiextensions.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
    - Resource 'customresourcedefinition/httpproxies.projectcontour.io (apiextensions.k8s.io/v1) cluster' is already associated with a different app 'contour-ctrl' namespace: tkg-system (label 'kapp.k14s.io/app=1693197485199497922')
  version: 1.20.2+vmware.2-tkg.1

As you can see from the above error same objects cannot be managed by two different app CR.

You would also see the app CR in the Reconciled Failed state.