Validating formatted IP Addresses/characters, in XFF Headers

Products

• ISG Proxy
• ProxySG Software - SGOS
• Advanced Secure Gateway Software - ASG

Issue/Introduction

The customer needs to check to verify that only IP addresses separated by commas appear in the XFF header, and that the IP addresses that appear are well constructed, and that no strange characters appear in the text string. How can this be done?

Environment

SG/ASG/ISG-Proxy

Resolution

Following investigation done, please be informed that there isn't an officially documented way to do the requested check.

Howbeit, to ensure that only properly formatted IP addresses appear in the X-Forwarded-For (XFF) header on a ProxySG (Edge SWG) and to check for the presence of only valid characters (i.e., digits, periods, and commas), you can use the following approaches:

1. Using CPL (Content Policy Language) in ProxySG
   
   ProxySG appliances allow administrators to use Content Policy Language (CPL) to create custom rules for handling HTTP traffic. You can create a CPL rule to inspect and validate the XFF header. Here's a basic outline of how this can be done:
   
   Creating a CPL Rule to Validate XFF Header
   
   
   1. 1. Extract the XFF Header:
         
         
         • Use CPL to extract the XFF header from the incoming HTTP request.
           
           
      2. Regex for IP Validation:
         
         
         • Use a regular expression (regex) to check that the header contains only valid IP addresses separated by commas. A typical regex for validating IPv4 addresses might look like this:
           
           ^(\d{1,3}\.){3}\d{1,3}(,\s*(\d{1,3}\.){3}\d{1,3})*$
           
           
      3. Policy Action
         
         
         1. If the XFF header matches the expected format, allow the request to proceed. If not, you can either remove the header, replace it, or log the event for further investigation.
            
            Sample Script
            

            define condition match_xff

            end

            <proxy>
              request.header.X-Forwarded-For="^(\d{1,3}\.){3}\d{1,3}(,\s*(\d{1,3}\.){3}\d{1,3})*$" condition=match_xff
            
            

            In this script:

            • The request.header.X-Forwarded-For directive checks the format of the XFF header
              
              
            • The condition=match part uses regex to verify that the header only contains IP addresses separated by commas.
              
              
            • If the condition is not met, the request can be denied or handled accordingly.
              
              
   2. Monitoring and Logging
      
      Use ProxySG's logging capabilities to monitor and log XFF header contents:
      
      
      1. Enable Detailed Logging:
         
         
         • Ensure that your access logs are configured to capture the XFF header. This can be done by customizing the log format in the Management Console or CLI.
           
           
      2. Log Analysis:
         
         
         • Regularly analyze logs to identify any anomalies or improperly formatted XFF headers. This can help you detect any issues that weren't caught by the CPL rules.
           
           
   3. Additional Tools and Methods
      
      
      1. External Scripting and Automation:
         
         
         • Consider using external scripts (e.g., Python) to periodically fetch logs from ProxySG, parse the XFF headers, and validate them against a regex pattern. This method can be used in environments where direct CPL scripting may not cover all needs.
           
           
      2. Network Monitoring Tools:
         
         
         • Use network monitoring tools like Wireshark to inspect the HTTP headers in real-time. This can help in debugging and verifying that only valid IPs are being logged.

So, by using CPL on the ProxySG to check the XFF header format, enabling detailed logging, and possibly using external tools for further validation, you can ensure that only valid IP addresses are present in the XFF header and that no unusual characters are introduced. This approach helps maintain the integrity and accuracy of client IP tracking, which is crucial for both security and analytics.