Exporting system logs for any virtual machine fails via vCenter UI and downloads an invalid file starting version 8.0u3.
search cancel

Exporting system logs for any virtual machine fails via vCenter UI and downloads an invalid file starting version 8.0u3.

book

Article ID: 376476

calendar_today

Updated On:

Products

VMware vCenter Server 8.0

Issue/Introduction

On attempting to download/export the system logs for any virtual machine:

Login to vCenter UI -> Right-Click any Virtual machine > Export System Logs,
it would export an invalid file of few KBs and fail immediately.

The export system logs option is greyed out for the VMs from the ESXi host UI.

Below is the screenshot for the same:

Similarly,  Exporting system logs for a virtual machine with "Send_NMI_To_Guest" option from vSphere Client results in downloading an invalid file that does not contain any data.

Below is the log snippet from the /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log:

[YYYY-MM-DD] [ERROR] nio-127.0.0.1-5090-exec-9007  c.v.vsphere.client.h5.admin.controllers.ExportLogsController
     FileId parameter is empty.
[YYYY-MM-DD] [ERROR] nio-127.0.0.1-5090-exec-9007  c.v.vsphere.client.h5.admin.controllers.ExportLogsController
     Error while downloading logs. java.io.IOException: Server returned HTTP response code: 500 for URL: http://localhost:1080
/external-vecs/443/cgi-bin/vm-support.cgi?manifests=
        at sun.reflect.GeneratedConstructorAccessor3066.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1952)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1947)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1946)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1516)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1500)
        at com.vmware.vise.util.http.ConnectionManager.getResponseBodyAsStream(ConnectionManager.java:341)
        at com.vmware.vise.util.http.SimpleHttpClient.executeMethodResponseAsStream(SimpleHttpClient.java:223)
        at com.vmware.vise.util.http.SimpleHttpClient.executeMethodResponseAsStream(SimpleHttpClient.java:244)
        at com.vmware.vsphere.client.logbundle.HttpZipEntry.connect(HttpZipEntry.java:211)
        at com.vmware.vsphere.client.logbundle.UrlZipEntry.write(UrlZipEntry.java:165)
        at com.vmware.vsphere.client.h5.admin.controllers.ExportLogsController.writeHostLogsToStream(ExportLogsController.java:608)
        at com.vmware.vsphere.client.h5.admin.controllers.ExportLogsController.access$500(ExportLogsController.java:82)
        at com.vmware.vsphere.client.h5.admin.controllers.ExportLogsController$1.run(ExportLogsController.java:372)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at com.vmware.vise.util.concurrent.ExecutorUtil$LatchRunnable.run(ExecutorUtil.java:1301)
        at com.vmware.vise.util.concurrent.ExecutorUtil.executeTasks(ExecutorUtil.java:325

Environment

VMware vCenter Server 8.0.3x

Cause

This is a known issue affecting the 8.0 U3 version of vCenter Servers.

Resolution

This issue is resolved in vCenter Server 8.0 Update 3e.

Note: Please note that suspending the VM and sending a hung command listed below may cause the guest operating system to crash temporarily.

Workaround:
Log in to the ESXi host through SSH or console as root and follow the steps below to generate the log bundles for specific VMs:

vm-support --listvms

Example output:
/vmfs/volumes/xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxxx/New Virtual Machine/New Virtual Machine.vmx (Registered)
/vmfs/volumes/xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxxx/Test VMachine/Test VMachine.vmx (Registered)

In the next step, select the complete path for the .vmx file for the VM from the above step:

vm-support --vm /vmfs/volumes/xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxxx/Test VMachine/Test VMachine.vmx

The VM logs will be generated, and on completion, it will output a path on the datastore as to where it is getting saved. 

In addition to the above, to customize the log bundle, i.e., suspending the VM, sending hung command, etc., then the manifests could be listed using the below command.

vm-support --listmanifests

vm-support -h could be used to check for all the available options.

Workaround for exporting system logs for a virtual machine with the "Send_NMI_To_Guest" option:

Use vm-support command-line to collect the System logs directly from the host:

 1. Log in to the ESXi host through SSH or the console as root.
Note: The virtual machine must be registered in the inventory.

 2. Run this command to list all registered and running virtual machines:

vm-support --listvms
vmdumper -l

 3. Run this command to send NMI:

vm-support -a HungVM:Send_NMI_To_Guest --vm=/vmfs/volumes/Path/of/VMname.vmx

Refer to KB: How to send NMI to Guest OS on ESXi 6.x or Later

Powered by Wolken Software