Multi-Line Log Messages are Not Displayed Correctly in Syslog (Snowflake Hunter) When Forwarding from VMware Aria Operations for Logs

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• When forwarding logs from VMware Aria Operations for Logs to an external syslog destination (Snowflake Hunter), multi-line messages are not displayed.
• Instead, each line of the original multi-line message appears as a separate log entry with different timestamps.
  • The first line retains the original timestamp from Aria Operations for Logs.
  • Subsequent lines are recorded with the syslog server’s timestamp.

Incorrect Result:

Expected Result:

Environment

Aria Operations for Logs 8.x

Cause

• Standard Syslog forwarding attempts to make logs RFC 5424 compliant, but the way multi-line payloads are handled can cause destination servers to interpret newlines as the start of a new message.
• While the payload is sent as a logical block, without Octet-based framing, the destination (Snowflake/Syslog) cannot correctly delineate where the multi-line event ends.

Resolution

Below command needs to be executed from the Aria Operations of logs appliance.

1. Take an ssh session to one of the Aria Operations for Logs appliance 
2. Get Token by running the following curl command 

   curl -X POST \       
     --url https://{LOGS_IP}:9543/api/v2/sessions \
     --header 'content-type: application/json' \
     --data '{
     "username": "admin",
     "password": {ADMIN_PASSWORD}
   }' 

3. Get info of the Log Forwarder record by running the following query

   curl -X GET \
     --url https://{LOGS_IP}:9543/api/v2/log-forwarder \
     --header 'authorization: Bearer {TOKEN}' \
     --header 'content-type: application/json' 

4. Then find the corresponding log forwarder and run the following API for updating the transport protocol:

   curl -X PUT \ 
     --url https://{LOGS_IP}:9543/api/v2/log-forwarder/{ID_OF_THE_LOG_FORWARDER_FROM_STEP_2} \
     --header 'authorization: Bearer {TOKEN}' \
     --header 'content-type: application/json' \
     --data '
     {
       "name": {GET_FROM_STEP_2_RESULT},
       "host": {GET_FROM_STEP_2_RESULT},
       "port": {GET_FROM_STEP_2_RESULT},
       "protocol": {GET_FROM_STEP_2_RESULT},
       "sslEnabled": {GET_FROM_STEP_2_RESULT},
       "workerCount": {GET_FROM_STEP_2_RESULT},
       "connectionRefreshInterval": {GET_FROM_STEP_2_RESULT},
       "diskCacheSize": {GET_FROM_STEP_2_RESULT},
       "tags": {GET_FROM_STEP_2_RESULT},
       "filter": {GET_FROM_STEP_2_RESULT},
       "transportProtocol": "tcp_octet",
       "forwardComplementaryFields": {GET_FROM_STEP_2_RESULT},
       "id": {GET_FROM_STEP_2_RESULT}
     }
   ' 

5. The syslog now should be able to see the full expected message.