ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.

AccessLog Parse Error with SGOS 7.3.7.1

book

Article ID: 240132

calendar_today

Updated On:

Products

ASG-S500 ProxySG Software - SGOS Reporter-S500 Reporter-VA

Issue/Introduction

After successful SGOS upgrade from  6.7.5.16 > 7.2.1.1 > 7.2.5.1 > 7.2.6.1 > 7.3.7.1., everything goes well. However, we discovered an issue with the Upload of some access logs, linked with WAF.

Below the definition of bcreporterwarp_v1 .. with 6.7.5.16 ... all is OK

psgdcv1-s2#sh access-log format bcreporterwarp_v1
Settings:
Format name: bcreporterwarp_v675
  Type elff "date time time-taken c-ip cs-username cs-auth-group x-bluecoat-transaction-uuid x-exception-id cs(Referer) sc-status s-action cs-method rs(Content-Type)cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-cs-client-ip-country x-user-x509-serial-number x-user-x509-subject rs-bytes x-cs-client-effective-ip x-cs-client-effective-ip-country cs(X-Forwarded-For) rs-service-latency r-ip x-bluecoat-application-name x-bluecoat-waf-attack-family x-risk-score x-bluecoat-waf-block-details x-bluecoat-waf-monitor-details x-bluecoat-request-details-header x-bluecoat-request-details-body x-bluecoat-waf-scan-info"
  Multiple-header-policy log-last-header

Below, is the updated definition for the same access log with 7.3.7.1.

psgdcv1-s5#sh access-log format bcreporterwarp_v1
Settings:
Format name: bcreporterwarp_v1
  Type elff "date time time-taken c-ip cs-username cs-auth-group x-bluecoat-transaction-uuid x-exception-id cs(Referer) sc-status s-action cs-method rs(Content-Type)cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id cs-threat-source cs-threat-id rs-threat-source rs-threat-id x-cs-client-ip-country x-user-x509-serial-number x-user-x509-subject rs-bytes x-cs-client-effective-ip x-cs-client-effective-ip-country cs(X-Forwarded-For) rs-service-latency r-ip x-bluecoat-application-name x-bluecoat-waf-attack-family x-risk-score x-bluecoat-client-address-reputation x-bluecoat-client-effective-address-reputation x-bluecoat-waf-block-details x-bluecoat-waf-monitor-details x-bluecoat-request-details-header x-bluecoat-request-details-body x-bluecoat-waf-scan-info "
  Multiple-header-policy log-last-header

This one is generating the below error with all Waf related issues with Reporter 10.6.2.1

Found 33 parse errors in 44 log lines of log source file 'ING-WAF:psgdcv2-s5-waf:/accesslogs/psgdcv2-s5-waf/SG_psgdcv2-s5_prptmza1-s5-waf_20220422114803.log.gz'

I tried to work around the issue by importing the 6.7 format definition on 7.3.7.1 but the issue persisted, with the attendant consequence of the lack of visibility for all Security events.

Cause

Having investigated the reported Accesslog "parse" errors, please refer to the below, for the cause of the issue.

Cause: This error is returned because the Reporter log source received a LIST reply that it did not understand. The Reporter service must be able to parse the contents of the response for known markers that help ensure the fields contain expected data. We see, in the updated log format, unknown, non-recommended, fields that the Reporter cannot parse, hence the error received.

Environment

Release: 7.3.7.1

Resolution

Resolution

To configure WAF Access Log, The bcreporterwarp_v1 access log format allows you to send data about reverse proxy-specific features (such as Geolocation and Application Protection) to Symantec Reporter. This is a reserved format and cannot be edited. The format includes the following access logging fields:

date time time-taken c-ip cs-username cs-auth-group x-bluecoat- transaction-uuid x-exception-id cs(Referer) sc-status s-action cs- method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri- path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs- bytes x-virus-id x-cs-client-ip-country x-user-x509-serial-number x- user-x509-subject rs-bytes x-cs-client-effective-ip x-cs-client- effective-ip-country cs(X-Forwarded-For) rs-service-latency r-ip x- bluecoat-application-name x-bluecoat-waf-attack-family x-risk-score x-bluecoat-waf-block-details x-bluecoat-waf-monitor-details x- bluecoat-request-details-header x-bluecoat-request-details-body x- bluecoat-waf-scan-info

Please refer to the Tech. doc. with the URL below, for the recommended details plus the description of the bcreporter_v1 Access Log Fields. Please note that these log fields are specific to SGOS 7.3.x.x.

https://techdocs.broadcom.com/us/en/symantec-security-software/web-and-network-security/proxysg/7-3/about-waf/chapter-3-configure-and-review-the-waf-access-log.html

For your updated log format, we see the below.

date time time-taken c-ip cs-username cs-auth-group x-bluecoat-transaction-uuid x-exception-id cs(Referer) sc-status s-action cs-method rs(Content-Type)cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id cs-threat-source cs-threat-id rs-threat-source rs-threat-id x-cs-client-ip-country x-user-x509-serial-number x-user-x509-subject rs-bytes x-cs-client-effective-ip x-cs-client-effective-ip-country cs(X-Forwarded-For) rs-service-latency r-ip x-bluecoat-application-name x-bluecoat-waf-attack-family x-risk-score x-bluecoat-client-address-reputation x-bluecoat-client-effective-address-reputation x-bluecoat-waf-block-details x-bluecoat-waf-monitor-details x-bluecoat-request-details-header x-bluecoat-request-details-body x-bluecoat-waf-scan-info

For the highlighted, non-recommended log fields, please see the additional information below.

cs-threat-source ==> Not a recognized log field for SGOS 7.3.x.x

cs-threat-id ==> Recognized log field for SGOS 7.3.x.x but not required for WAF access logging. This log field is the "Identifier of the threat if detected by REQMOD ICAP."

x-bluecoat-client-address-reputation ==> Not a recognized log field for SGOS 7.3.x.x

x-bluecoat-client-effective-address-reputation ==> Not a recognized log field for SGOS 7.3.x.x

Ref. doc.: https://techdocs.broadcom.com/content/dam/broadcom/techdocs/symantec-security-software/web-and-network-security/proxysg/common/LogFieldsSubs.pdf

Please utilize only the recommended access log format, specifically reserved for the WAF access log. It's not to be edited. This way, the Reporter would be able to parse the log lines.