Enable Debug Logging for HCX OS-Assisted Migration Troubleshooting
search cancel

Enable Debug Logging for HCX OS-Assisted Migration Troubleshooting

book

Article ID: 386461

calendar_today

Updated On:

Products

VMware HCX

Issue/Introduction

When troubleshooting OS-Assisted Migration (OSAM) issues in HCX, enabling debug logging provides detailed insights into the migration process. Debug logs are essential for

  • Analyzing OSAM migration and replication processes
  • Investigating communication issues between HCX components
  • Troubleshooting Sentinel Gateway and Data Receiver operations
  • Diagnosing failed/stalled migrations

Environment

  • VMware HCX Enterprise
  • HCX Connector
  • HCX Cloud Manager
  • HCX Sentinel Data Receiver (SDR)

Cause

Standard logging levels does not always provide sufficient detail for advanced troubleshooting of OSAM-related issues. Debug-level logging must be enabled across multiple HCX components to capture comprehensive diagnostic information.

Resolution

HCX Debug Logging Configuration Guide

IMPORTANT NOTICE

  • Before proceeding with debug logging configuration, ensure you have a plan to disable debug logging after completing your troubleshooting activities.
  • If you have the necessary permissions, create a snapshot of the virtual machine before making any changes.
  • Debug logging significantly increases system resource usage and log storage consumption.

Enable Debug Logging
-------------------

1. HCX Manager Debug Logging (Connector and Cloud)

Access the HCX Manager

  • Connect via SSH to the HCX Manager using administrator credentials
  • Execute: `ssh admin@<hcx-manager-ip>`

Modify the logging configuration

  • Edit the configuration file: `sudo vi /opt/vmware/config/app-logback.xml`
  • Add the following debug loggers before the closing tag

<logger name="com.vmware.vchs.hybridity.service.migration" additivity="false" level="DEBUG">
    <appender-ref ref="APP_FILE"/>
    <appender-ref ref="SYSLOG"/>
</logger>
<logger name="com.vmware.vchs.hybridity.service.os_assisted_replication" additivity="false" level="DEBUG">
    <appender-ref ref="APP_FILE"/>
    <appender-ref ref="SYSLOG"/>
</logger>
<logger name="com.vmware.vchs.hybridity.service.sentinelinventory" additivity="false" level="DEBUG">
    <appender-ref ref="APP_FILE"/>
    <appender-ref ref="SYSLOG"/>
</logger>
<logger name="com.vmware.vchs.hybridity.adapter.sentinelgateway" additivity="false" level="DEBUG">
    <appender-ref ref="APP_FILE"/>
    <appender-ref ref="SYSLOG"/>
</logger>
<logger name="com.vmware.vchs.hybridity.adapter.sentineldatareceiver" additivity="false" level="DEBUG">
    <appender-ref ref="APP_FILE"/>
    <appender-ref ref="SYSLOG"/>
</logger>

  • Save the file using :w! in vi
  • Restart the service: sudo systemctl restart app-engine

3. HCX Sentinel Data Receiver Debug Logging

Access the SDR appliance

  • SSH to HCX Cloud Manager: ssh admin@<hcx-cloud-manager-ip>

  • Locate SDR appliance: ccli list go <sdr-id> ssh

Configure journald logging

  • Edit: sudo vi /etc/systemd/journald.conf

  • Set: MaxLevelSyslog=debug

  • Restart service: sudo systemctl restart systemd-journald

Configure SDR logging

  • Edit: sudo vi /opt/vmware/sdr/config/sdrd.conf

  • Set the loglevel to debug in the configuration

{
    "loglevel": "debug"
}

  • Restart service: sudo systemctl restart sdrd

Revert Debug Logging
-------------------

After completing troubleshooting, follow these steps to revert debug logging:

1. Revert HCX Manager Logging

Access the HCX Manager

  • Connect via SSH to the HCX Manager: ssh admin@<hcx-manager-ip>

Modify the logging configuration

  • Edit the configuration file: sudo vi /opt/vmware/config/app-logback.xml

  • Locate and remove all the debug logger entries added previously, or change their level from "DEBUG" to "INFO"

  • Save the file using :w! in vi

  • Restart the service: sudo systemctl restart app-engine

2. Revert SDR Logging

Access the SDR appliance through HCX Cloud Manager

  • SSH to HCX Cloud Manager: ssh admin@<hcx-cloud-manager-ip>

  • Access SDR: ccli list go <sdr-id> ssh

Revert journald logging

  • Edit: sudo vi /etc/systemd/journald.conf

  • Change: MaxLevelSyslog=debug to MaxLevelSyslog=info

  • Restart service: sudo systemctl restart systemd-journald

Revert SDR logging

  • Edit: sudo vi /opt/vmware/sdr/config/sdrd.conf

  • Change the loglevel back to info

{
    "loglevel": "info"
}

  • Restart service: sudo systemctl restart sdrd

If issues persist after enabling debug logging, contact Broadcom Support with the following information

  • Migration details (VM Name, Migration ID, UI error screenshots)
  • Source and Target HCX log bundles including HCX database dumps and IX appliance logs
  • Debug logs from the relevant components enabled above

Additional Information

Sub-Components for Debug Logging

Component Location Purpose
service.migration Both HCX-Connector and HCX-Cloud Debugging OSAM migration-related issues
service.os_assisted_replication Both HCX-Connector and HCX-Cloud Debugging OSAM replication-related issues
service.sentinelinventory HCX-Connector only Debugging Sentinel Inventory service
adapter.sentinelgateway HCX-Connector only Debugging communication between HCX-Connector and SGW
sentineldatareceiver HCX-Cloud only Debugging communication between HCX-Cloud and SDR
 
Powered by Wolken Software