Configuring syslog on NSX-T node using cli fails "an error occurred while adding the logging-server"

Products

• VMware NSX

Issue/Introduction

• Syslog was configured in NSX-T using the System -> Fabric -> Profiles -> Node Profiles
• The Node Profile was then removed and NSX CLI was used to add the syslog configuration on the nodes.
• When issuing a syslog configuration command such as:

set logging-server <server IP> proto udp level info structured-data audit="true"

• Below error was reported after running the command:

"% an error occurred while adding the logging-server"

• In the NSX manager logs, following error was reported:

[nsx@6876 comp="nsx-manager" subcomp="cli" username="admin" level="ERROR" errorCode="('CLI110',)"] Error setting logging server: {'error_code': 36125, 'error_message': 'Error reading syslog exporter properties.','module_name': 'node-services'}

• Central config is enabled on the nodes, which can be validated by running the below command:

get node central-config

Environment

VMware NSX

Cause

When node central config is enabled (it is enabled by default on all nodes), the local configuration will be over written on the node at a frequency up to every 4 hours, this is expected behavior.
This issue occurs due to central-config getting stuck in a loop checking desired and realized state.

Resolution

This issue has been resolved in VMware NSX-T Data Center 3.2.1, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Workaround: 

As admin user, disable central-config on the nodes (has to be done manually on each node) and then apply the syslog configuration on the nodes, as this command will prevent the node from accepting the node profile's central configuration:

set node central-config disabled

To check the status of the central-config please run the command:

get node central-config

Additional Information

For more information regarding the configuration of syslog server on NSX nodes from the UI, please refer the article: Add Syslog Servers for NSX Nodes