Main HTTP and HTTPS access log fields required for optimal performance using Reporter

Products

• Reporter

Issue/Introduction

The Reporter software expects to see these fields in the access log, for the accuracy of reporting and efficiency. Symantec recommends using logs that conform to ELFF standards and only contain the following fields.
 

 

Resolution

Using a Secure Gateway appliance from Symantec, you can choose these named access logs to ensure your HTTP and HTTPS access logs conform:

• For main HTTP logs, choose the access log named bcreportermain_V1.
• For main HTTPS logs, choose the access log named bcreporterssl_v1.
• For video streaming activity we can now choose bcreporterstreaming_v1  (only available in SGOS 6.2.X releases and later).
• On occasions, you might need to create your own access logs. To ensure you have trouble free reporting, refer to the fields outlined below.

The fields in HTTP main logs:

date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bluecoat-application-name x-bluecoat-application-operation x-bluecoat-application-groups cs-threat-risk x-bluecoat-transaction-uuid x-icap-reqmod-header(X-ICAP-Metadata) x-icap-respmod-header(X-ICAPMetadata)

The fields in HTTPS main logs:

date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id scfilter-result cs-categories sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-rs-certificate-observederrors x-cs-ocsp-error x-rs-ocsp-error x-rs-connection-negotiatedcipher-strength x-rs-certificate-hostname x-rs-certificate-hostnamecategory cs-threat-risk x-rs-certificate-hostname-threat-risk

Notes:

1. HTTPS logs do not contain the cs(Referer) field; therefore, the PVC process cannot occur. The field is not included because it would expose personal user data (such as bank account information). Reporter not guarantee to work PVC if force input cs(Referer) field for HTTPS logs.
2. The cs-uri-query,cs-uri-path to the SSL access logs might inadvertently expose sensitive user data to the access logs, such as user names and passwords. Typically this data would be encrypted but if the Edge SWG (ProxySG) is doing SSL interception it will unencrypt the contents and write the results to the access logs which will then be visible in clear text."

The fields in the new video streaming logs- bcreporterstreaming_v1:

date time time-taken c-ip sc-status s-action sc-bytes rs-bytes cs-method cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-username cs-auth-group cs(Referer) cs(User-Agent) c-starttime filelength filesize avgbandwidth x-rs-streaming-content x-streamingrtmp-app-name x-streaming-rtmp-stream-name x-streaming-rtmp-swf-url x-streaming-rtmp-page-url s-ip s-dns s-session-id x-cache-info

While Symantec does not recommend varying from the lists provided above, some fields are perhaps more essential than others.

For core databases functionality:

cs-host, sc-status, cs-uri-scheme

Notes:

1. Reporter can't display as report data if these fields not contain specific data.
2. Reporter is able to input all accesslog fields in database but there is some fields not display as report data such a "x-bluecoat-transaction-uuid".

"x-bluecoat-transaction-uuid" means an object identifier and sample value is "b99f0889f8d22eda-000000000002b7c5-000000005e4e292b".

For the Page view combiner feature (PVC):

cs(Referer) or x-cs(Referer)-uri
x-exception-id, (or sc-filter-result),
sc-filter-category, cs-category, or cs-categories

For Dashboard reports that are configured by default:

cs-username, cs-user, x-cache-user, cs-userdn, x-radius-splash-username, or x-cs-session-username

Note: You need only one of the user based fields.  

When using HTTPS Main logs:

x-rs-certificate-observed-errors (Certificate Error)
x-rs-certificate-hostname (Cert Svr Domain)
x-rs-certificate-hostname-category (Certificate Category)
x-rs-connection-negotiated-cipher-strength (Cipher Strength)