Enabling debug logging for SSO and Authentication services in vCenter Server

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article provides steps to enable debug logging for SSO and other authentication-related services in vCenter Server for troubleshooting purposes.

Environment

VMware vCenter Server 8.0.x
VMware vCenter Server 7.0.x

Resolution

Important: Ensure that you back up the original configuration files and that the changes are reverted to INFO after collecting the data to avoid performance impacts.

In vSphere 7.0 and 8.0, many of the SSO components are combined under shared root loggers. For example, the log files under /var/log/vmware/sso/ share a root logger that contains the following components.

STS	/var/log/vmware/sso/vmware-identity-sts.log /var/log/vmware/sso/vmware-identity-sts-perf.log
IDM	/var/log/vmware/sso/vmware-rest-idm.log
AFD	/var/log/vmware/sso/vmware-rest-afd.log
VMDIR	/var/log/vmware/sso/vmware-rest-vmdir.log
OpenID Connect	/var/log/vmware/sso/openidconnect.log
WebSSO	/var/log/vmware/sso/websso.log
SSO Admin Service	/var/log/vmware/sso/ssoAdminServer.log
Token Service	/var/log/vmware/sso/tokenservice.log
Active Directory Service	/var/log/vmware/sso/activedirectoryservice.log
Service Account Management	/var/log/vmware/sso/svcaccountmgmt.log
HTTP Wire	/var/log/vmware/sso/http-wire.log
Audit Event Log	/var/log/vmware/sso/audit_events.log
Operation Events	/var/log/vmware/sso/operation_events.log

1. To enable debug level logging across all of these components, edit the shared log4j2.xml file.

vSphere 7.0: /usr/lib/vmware-sso/vmware-sts/webapps/ROOT/WEB-INF/classes/log4j2.xml

vSphere 8.0: /var/lib/sso/webapps/ROOT/WEB-INF/classes/log4j2.xml

2. Locate the Root level section near the botttom of the file:

3. Edit the line:

The configuration should take effect without a restart of the services

Possible logging options for this section are: trace, debug, info, warn, error, fatal

VMDirectory Service (vmdir)

VMware Directory Service

/var/log/vmware/vmdird/vmdird-syslog

1. To enable verbose logging for VMdir, use the CLI utility vdcadmintool.

/usr/lib/vmware-vmdir/bin/vdcadmintool

2. Choose option 4 and set the log level 4 "DEBUG" with a Log mask of "0"

==================
Please select:
0. exit
1. Test LDAP connectivity
2. Force start replication cycle
3. Reset account password
4. Set log level and mask  
5. Set vmdir state
6. Get vmdir state
7. Get vmdir log level and mask
==================

4
  Log level (ERROR|WARNING|INFO|VERBOSE|DEBUG): DEBUG
  Log mask : 0

No restart of services is necessary.

To restore the default log/mask, run the same tool and set the log level to INFO and the mask to 0

Vpxd-svcs and Authz

VMware VPXD Services	/var/log/vmware/vpxd-svcs/vpxd-svcs.log
Authz Service	/var/log/vmware/vpxd-svcs/authz-event.log

The vpxd-svcs and authz-event logs share a log4j properties file.

/usr/lib/vmware-vpxd-svcs/lib/server/config/log4j.properties

1. Create a backup copy of the properties file.

cp -p /usr/lib/vmware-vpxd-svcs/lib/server/config/log4j.properties /root/vpxd-svcs_log4j.properties.bak

2. Edit the properties file and change the following lines from INFO to DEBUG

log4j.rootLogger=DEBUG, LOGFILE, syslog
log4j.appender.syslog.threshold=DEBUG
log4j.logger.com.vmware.cis=DEBUG
log4j.appender.LOGFILE.Threshold=DEBUG

3. Restart the vpxd-svcs service.

service-control --restart vpxd-svcs

Trust Management Service (vmware-trustmanagement)

Trust Management Service

/var/log/vmware/trustmanagement/trustmanagement-svcs.log

Trust Management uses the following properties file:

/usr/lib/vmware-trustmanagement/config/log4j.properties

1. Stop the vmware-trustmanagement service

service-control --stop vmware-trustmanagement

2. Create a backup copy of the properties file.

cp -p /usr/lib/vmware-trustmanagement/config/log4j.properties /root/trustmanagment_log4j.properties.bak

3. Edit the properties file and update the root logger from INFO to DEBUG

# Root logger option
log4j.rootLogger=DEBUG, file

4. Restart the vmware-trustmanagement service

service-control --start vmware-trustmanagement

Likewise

Likewise consists of several daemons that are used to interact with Active Directory domains. These each have their own purposes and log levels.

Use the commands below to set the log level for each of these daemons from info to trace. Ensure these are reverted once the required information is collected.

lwsmd: /opt/likewise/bin/lwsm set-log-level trace
lwiod: /opt/likewise/bin/lwio-set-log-info trace
netlogond: /opt/likewise/bin/lwnet-set-log-level trace
lsassd: /opt/likewise/bin/lw-set-log-level trace

By default, all likewise logs are sent to /var/log/vmware/messages.

For more information on enabling likewise logging, see Modifying the log level of the Likewise agent in VMware vCenter Server Appliance

For more information on what the purpose of each likewise daemon is, see Enabling logging for Likewise agents on ESXi/ESX