"Namespace is not ready" or "Auth Failure" during Aria Operations Service Discovery with VMware Tools 13.x and 12.5.x

Products

• VCF Operations
• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Symptoms

Users may observe one or more of the following symptoms:

• Discovery Failures: Service discovery does not initiate, terminates prematurely, or results in incomplete service data (e.g., only a subset of expected servers is discovered).
• Authentication Status: The VM Authentication Status in Aria Operations unexpectedly changes to "Failed" or "Unknown" after a tools upgrade or reboot.
• vCenter Events: The vCenter Tasks and Events tab for the affected VM displays the following warning:
  • com.vmware.vc.guestOperations.GuestOperationAuthFailure
• Adapter Logs: The ServiceDiscoveryAdapter logs on the Aria Operations node (or Cloud Proxy) contain the following exception:
  • com.vmware.adapter3.applicationdiscovery.guestoperation.exception.NamespaceIsNotReady: Namespace is not ready.

Environment

VCF Operations 9.0.x
Aria Operations 8.18.x

Cause

This issue is caused by a synchronization failure between the VMware Tools SDMP (Service Discovery Management Platform) plugin and the Aria Operations Service Discovery Adapter.

When VMware Tools is upgraded to version 13.0.5 or is running certain 12.5.x builds, the com.vmware.vrops.sdmp namespace in the vCenter Managed Object Browser (MOB) can become stale or improperly formatted. This prevents the Guest Operations Manager from re-initializing the credential-less handshake. As a result, the adapter cannot read the required service data from the guest OS, leading to the "Namespace is not ready" exception and a fallback to authentication failure.

Resolution

It's advisable to take snapshots of vCenter VM as per Snapshot Best practices for vCenter Server Virtual Machines before proceeding with steps below:

Steps to delete com.vmware.vrops.sdmp from vCenter MOB:

1. Open a browser and navigate to:
   
   https://<vcenter-ip>/mob/?moid=namespaceManager&vmodl=1
   
   
2. From the list of methods, select deleteNamespace
3. In the input fields, provide:
   
   vm: MOID of the target VM (this can be retrieved from the Virtual Machine Object properties)
   Name: com.vmware.vrops.sdmp
4. Scroll down and click Invoke Method
5. If the deletion is successful, the response will show:
   Method Invocation Result: void
6. Restart the Service Discovery Adapter Instance from Aria Operations Manager

Additional Information

In case there are more than a single VM that needs to be fixed, you can also run below script : delete_sdmp_namespaces.py

It's advisable to take snapshots of vCenter VM as per Snapshot Best practices for vCenter Server Virtual Machines before proceeding with steps below:

To run the script follow below steps : 

1. Copy the script on a machine that has Python installed
2. Run following command : python delete_sdmp_namespaces.py
3. It will prompt:
   1. Host destination, provide FQDN for vCenter
   2. Username, provide vCenter username
   3. Password, provide vCenter password
4. If it is successful It will show 'Done'

Note : In case of Error: 'requests' module not found, install requests module by running pip install requests