How to recover Data Services Manager (DSM) Consumption Operator (9.0.x versions) from 'MANIFEST_UNKOWN' error
search cancel

How to recover Data Services Manager (DSM) Consumption Operator (9.0.x versions) from 'MANIFEST_UNKOWN' error

book

Article ID: 413248

calendar_today

Updated On:

Products

VMware Data Services Manager

Issue/Introduction

DSM Consumption Operator is showing a ReconcileFailure error due to missing images in the customer artifactory (packages.broadcom.com)

Environment

DSM 9.0.x

Cause

The registry location of DSM Consumption Operator is changed in the customer artifactory 

Version

Old Image Location

Old Artifact Location

New Image Location New Artifact Location

9.0.1.0.24913742

projects.packages.broadcom.com/dsm-consumption-operator/consumption-operator:9.0.1.0.24913742
projects.packages.broadcom.com/dsm-consumption-operator/dsm-consumption-operator-supervisor:9.0.1.0.24913742

https://packages.broadcom.com/artifactory/dsm-distro/dsm-consumption-operator/supervisor-service/9.0.1.0/package.yaml
https://packages.broadcom.com/artifactory/dsm-distro/dsm-consumption-operator/supervisor-service/9.0.1.0/values.yaml

projects.packages.broadcom.com/vsphere/dsm-consumption-operator/consumption-operator:9.0.1.0.24913742
projects.packages.broadcom.com/vsphere/dsm-consumption-operator/dsm-consumption-operator-supervisor:9.0.1.0.24913742
https://support.broadcom.com/group/ecx/productfiles?subFamily=vSphere%20Supervisor%20Services&displayGroup=Data%20Services%20Manager%20Consumption%20Operator&release=9.0.1.0&os=&servicePk=535766&language=EN

9.0.0.0.24665381

projects.packages.broadcom.com/dsm-consumption-operator/consumption-operator:9.0.0.0.24665381
projects.packages.broadcom.com/dsm-consumption-operator/dsm-consumption-operator-supervisor:9.0.0.0.24665381

 

https://packages.broadcom.com/artifactory/dsm-distro/dsm-consumption-operator/supervisor-service/9.0.0.0/package.yaml
https://packages.broadcom.com/artifactory/dsm-distro/dsm-consumption-operator/supervisor-service/9.0.0.0/values.yaml

projects.packages.broadcom.com/vsphere/dsm-consumption-operator/consumption-operator:9.0.0.0.24665381
projects.packages.broadcom.com/vsphere/dsm-consumption-operator/dsm-consumption-operator-supervisor:9.0.0.0.24665381

https://support.broadcom.com/group/ecx/productfiles?subFamily=vSphere%20Super

 

Resolution

Follow the below steps to recover the DSM Consumption Operator from the above mentioned error.

Copy the service config:

Copy the service config values so that you can re-use it when reinstalling DSM Consumption Operator

  • Go to Services under Supervisor Management
  • On the DSM Consumption Operator service card, click Actions, then select Manage Service
  • Choose the desired supervisor, and click Next
  • On the Validation page, click Next to proceed
  • On the Review page, copy the YAML service configuration values and save them for future reference

Finally, click Cancel on the Review page.


Uninstall the current DSM Consumption Operator supervisor service:

  • Go to Services under Supervisor Management
  • On the DSM Consumption Operator supervisor service card, click Actions, and then choose Delete
  • Click Confirm to deactivate the service
  • Click Confirm again to deactivate all service versions across supervisors
  • Click Confirm once more to delete all versions of the service
  • Click the Delete button to remove the service completely
  • Wait until the svc-consumption-operator-domain-x Namespace is fully deleted

Reinstall the DSM Consumption Operator supervisor service by deploying the updated package.yaml package:

  • Obtain the appropriate version of the DSM Consumption Operator from the Broadcom Support Portal
    • Log in to the Broadcom Support Portal
    • Navigate to My Downloads. Use the filter option to display vSphere Supervisor Services
    • From the list, select Data Services Manager Consumption Operator
    • Choose the correct version of the DSM Consumption Operator for your environment
    • Download both the package.yaml and values.yaml files
  • Return to the vSphere Client
  • Go to Services under Supervisor Management
  • Click the Add button on the Add New Service card
  • Click Upload and select the package.yaml file downloaded in the previous step
  • Click Finish to deploy the service
  • Once the DSM Consumption Operator card appears on the Services page, click Actions and select Manage Service

  • On the Configure page, choose the desired install version and supervisor, then click Next
  • On the Review page:
    • Merge the previously copied service configuration values with the values.yaml file downloaded earlier
    • ⚠️ Do not overwrite the image and tag values from the values.yaml file — these represent the updated image locations. The old image references from the previous configuration are no longer valid
    • Remove the namespace entry from the copied values, as a new namespace will be auto-generated during deployment
    • Paste this new config in the YAML Service Config
  • Navigate to Supervisor → Configure to monitor the deployment status

Additional Information

Note: 
If you encounter an error during reinstallation where the image is still being pulled from projects.packages.broadcom.com/dsm-consumption-operator instead of projects.packages.broadcom.com/vsphere/dsm-consumption-operator, the issue may be caused by image caching in the kapp-controller. Follow the below steps,

  • Allow some time to pass between the uninstallation and reinstallation of the service
  • You can also restart the kapp-controller on the supervisor cluster by executing the following command:
     
    kubectl rollout restart deployment/kapp-controller -n vmware-system-appplatform-operator-system