Using the 'lsdoctor' Tool
search cancel

Using the 'lsdoctor' Tool

book

Article ID: 320837

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Lookup Service Doctor (lsdoctor) is a tool used to address issues with data stored in the PSC database, as well as data local to a vCenter (regardless of whether the PSC is external or embedded).  The tool can be used to detect and correct problems that could cause failures in topology changes (converge, repoint, etc.), upgrades, or failures incurred as a result of maintenance (e.g. incorrectly applying new SSL certificates).  This article will outline its functions and use.


Environment

VMware vCenter Server 6.7 with external Platform Service Controller. 

VMware vCenter Server 6.7 

VMware vCenter Server 7.x

VMware vCenter Server 8.x

Currently, lsdoctor supports vCenter Server 6.7 (Windows-based or VCSA) and above. When new builds of vCenter are released, lsdoctor must be updated asynchronously.  This means that lsdoctor support for the latest version of vCenter may be updated sometime after a new build is released.

Resolution

WARNING

Before using lsdoctor to make any changes, ensure you have taken proper snapshots of your SSO domain. This means that you must shut down all VCs or PSCs that are in the SSO domain at the same time, then snapshot them, and power them on again.  If you need to revert to one of these snapshots, shut all the nodes down, and revert all nodes to the snapshot. Failure to perform these steps will lead to replication problems across the PSC databases.

 ----

Installation

To use lsdoctor, you must download the ZIP file attached to this article.  Then, use the file-moving utility of your choice (WinSCP for example) to copy the entire ZIP directory to the node on which you wish to run it.

NOTE:  If you have troubles connecting to a vCenter appliance using WinSCP, please see Error when uploading files to vCenter Server Appliance using WinSCP


Once the tool is copied to the system, unzip the file:
Windows:

Right-click the file and click “Extract All…”

VCSA

Change your directory to the location of the file, and run the following command:
unzip lsdoctor.zip


NOTE:  When running the tool, be sure you are currently in the “lsdoctor-main” directory.
 

Launching the Tool

First, ensure you are in the lsdoctor-main directory from a command line (SSH session for VCSA).
To run lsdoctor, use the following command:

VCSA

python lsdoctor.py --help


 
Windows-Based vCenter
 
  1. Open PowerShell
  2. Navigate to the vCenter's Python directory:
    • cd C:\Program Files\VMware\vCenter Server\python
  3. Execute the lsdoctor.py script: 
    • .\python.exe <path_to_lsdoctor.py_file> <switch>
      • Example of running a Rebuild action: .\python.exe C:\Users\Administrator\Desktop\lsdoctor\lsdoctor.py -r
Functions

Skip to function:

 

-l, --lscheck

This option checks for common issues in the lookup service.  Does not make any changes to the environment.  This will show issues found on any node in the SSO domain.  See output for findings and path to JSON report.


Instructions:

  1. copy and extract lsdoctor to the filesystem of any node
  2. Run “python lsdoctor.py -l”
  3. Provide the password for your SSO administrator account
  4. Review output for issues found.  Each node will be represented by it’s SSO site followed by its hostname or PNID

Follow up actions needed:

  • Follow output instructions if applicable.

-p, --pscHaUnconfigure

This option is used when removing a PSC HA configuration (multiple external PSCs behind a load balancer).  To use this option, follow these steps:


Instructions:

  1. For each PSC behind the load balancer:
    1. copy and extract lsdoctor to the filesystem
    2. Run “python lsdoctor.py -p”
    3. Verify that you have taken the appropriate snapshots
    4. Provide the password for your SSO administrator account
    5. Once the script completes, move to the next node behind the load balancer.

Follow up actions needed:

  • Once lsdoctor has run on all nodes behind the LB (all PSCs in the SSO site), restart services on all of the PSCs on which you have run the script
  • Repoint your vCenter server(s) to the node of your choice.  For more information, see How to repoint vCenter Server 6.x between External PSC within a site.
  • Re-register any external solutions that were previously pointed to the load balancer VIP (SRM, vSphere Replication, NSX-V, etc. – See product documentation for instructions)

-s, --stalefix

This option cleans up any stale configurations left over from a system upgraded from 5.x.


Instructions:

  1. copy and extract lsdoctor to the filesystem of the affected node
  2. Run “python lsdoctor.py -s”
  3. Verify that you have taken the appropriate snapshots
  4. Provide the password for your SSO administrator account

Follow up actions needed:

  • Once the script completes, restart all services
  • Re-register any external solutions that were previously pointed to the affected node (SRM, vSphere Replication, NSX-V, etc. – See product documentation for instructions)

-t, --trustfix

This option corrects SSL trust mismatch issues in the lookup service.  The lookup service registrations may have an SSL trust value that doesn’t match the MACHINE_SSL_CERT on port 443 of the node.  This can be caused by a failure during certificate replacement, among other failures.


Instructions:

  1. copy and extract lsdoctor to the filesystem of any node in the same SSO site as the affected node(s)
  2. Run “python lsdoctor.py -t”
  3. Verify that you have taken the appropriate snapshots
  4. Provide the password for your SSO administrator account

Follow up actions needed:

  • Once the script completes, restart all services on all nodes in the SSO site
  • Re-register any external solutions that were previously pointed to the affected node(s) (SRM, vSphere Replication, NSX-V, etc.)

-u, --solutionusers

This option is used to recreate solution users for a node.  There are many reasons a solution user may be missing or inconsistent, but this script will delete any existing entry and recreate them from scratch.


Instructions:

  1. copy and extract lsdoctor to the filesystem of the affected node
  2. Run “python lsdoctor.py -u”
  3. Verify that you have taken the appropriate snapshots
  4. Provide the password for your SSO administrator account
  5. Once the script completes, restart all services on this node

Follow up actions needed:

  • None

-r, --rebuild

This option is used when rebuilding one or all service registrations for a given node.  This action performs the most significant changes, and extra care should be taken to ensure a safe rollback is possible (see Impact/Risks section).

 

Instructions:

  1. copy and extract lsdoctor to the filesystem of any node in the same SSO site as the affected node(s)
  2. Run “python lsdoctor.py -r”
  3. Verify that you have taken the appropriate snapshots
  4. Provide the password for your SSO administrator account
  5. You will be presented with a menu:
NOTE:  Version, Deployment type, and hostname references will be unique to your environment.


Option 1:

Option 1 is only used when lsdoctor does not have the appropriate files for your build of vCenter.  This will generate the file for your build (see output for path to the template).  You can then copy the template generated from a healthy system to an unhealthy system of the same build for use with the other options.  This should rarely be needed.


Option 2:

 Option 2 will take a backup of the existing lookup service registrations for this node, remove them, and register new ones based on a template.  Most templates are provided in the templates directory, and you should only need to select a template if it doesn’t exist.  If the template for your build of vCenter does not exist, you will be prompted to select one.  You should try to find the template that most closely matches your system.  Alternatively, you can use Option 1 on a healthy system of the same build and copy the template to the affected system.

Note:
If lsdoctor -r option 2 is failing to create a missing service registration, the following steps can be attempted.

1.Run lsdoctor -r option 2 and take note of the template version lsdoctor is attempting to use.
2.Then navigate into the templates directory in the lsdoctor folder and move out the template from this folder.
3.Re-run lsdoctor -r option 2 this time it will give a warning that a matching template cannot be found and to select from a template list.
4.At this stage select the template of closest matching build number.


Option 3:

Option 3 will take a backup of all lookup service registrations for this node, load the template for your current build, and provide a service select menu.  Select a service type you wish to replace, and the tool will unregister all services for this node with the selected service type and register a fresh one based on the detected template.  As with Option 2, you may be prompted to select a template if one is not automatically provided.


Option 4:

Option 4 allows you to restore the lookup service registrations from an automatic backup taken by performing option 2 or option 3.  You will be prompted to select a backup that is marked by timestamp (oldest to newest).  It will then remove all current service registrations and register the ones from the backup.


Follow up actions needed:

  • Once the script completes, restart all services
  • Re-register any external solutions that were previously pointed to this node (SRM, vSphere Replication, NSX-V, etc.)



Additional Information

VMware Skyline Health Diagnostics for vSphere - FAQ


Glossary

Node – Refers to a vCenter Server with embedded Platform Services Controller, an external Platform Services Controller, or a vCenter server pointed to an external Platform Services Controller

SSO domain -- Each vCenter server (or PSC if external) is associated with a vCenter Single Sign-On domain. The domain name defaults to vsphere.local, but you can change it during installation of the first Platform Services Controller. The domain determines the local authentication space.  All nodes in Enhanced Linked Mode are part of the same SSO domain

SSO site -- You can split an SSO domain into multiple sites and assign each embedded vCenter server or external Platform Services Controller and external vCenter Server instance to a site. Sites are logical constructs that group nodes together, but usually correspond to geographic location.

 

 

The script will fail with below error if /etc/issue file is update with a custom login banner. In such scenarios, the file should be updated to the default content as below (The version must be updated according to the actual version of the vCenter) before running the script and revert back later as desired.


VMware vCenter Server 8.0.3.00000

Type: vCenter Server with an embedded Platform Services Controller

 

Error

[ ~/lsdoctor-240201 ]# ./lsdoctor.py -l
Traceback (most recent call last):
  File "/root/lsdoctor-240201/./lsdoctor.py", line 547, in <module>
    main()
  File "/root/lsdoctor-240201/./lsdoctor.py", line 458, in main
    params = parameters()
  File "/root/lsdoctor-240201/./lsdoctor.py", line 136, in parameters
    params = get_params()
  File "/root/lsdoctor-240201/lib/utils.py", line 1349, in __init__
    version, build = _getVersion()
  File "/root/lsdoctor-240201/lib/utils.py", line 145, in _getVersion
    version = version.rsplit(' ',1)[1]
IndexError: list index out of range

 


Impact/Risks:


LIMITATIONS


  

HASH Info:

The hashes listed are only valid for the current files. 
Date hash added 7/25/2024
If the file is updated a new hash will need to be added

The current verson is lsdoctor-240201.zip
MD5SUM: 10069b3148747e4361d0867fc6d8783f
SHA256SUM: 5562545eb6d2369e80d3a46115f39b6bf4f03b2e1cd772b8f73a4efda6707be9

Attachments

lsdoctor-240201 get_app
Powered by Wolken Software