Importing a large list of URLs into Policy
search cancel

Importing a large list of URLs into Policy

book

Article ID: 278187

calendar_today

Updated On:

Products

ISG Proxy ProxySG Software - SGOS Management Center Management Center - VA Advanced Secure Gateway Software - ASG

Issue/Introduction

What are the options available to bulk update URLs in the EdgeSWG policy?

Resolution

 
Info and resources on each option:

 

Shared Objects

Using a Shared Object in Management Center such as an URL List or CPL Fragment and then associating the shared object in the VPM policy and adding the Object to Policy can be done.

With a shared object, the URL's can be added in bulk, and without needing to update the Policy the included shared object will automatically be updated.

Create Management Center URL List (URL Policy Exceptions) 

Add a Shared Object to MC VPM Policy

 

Policy Import to MC

The concept of Shared Objects can be extended to hosting the URL list on a server and the MC can download these files and update the Shared Object using jobs. Hosting the file on MC directly would provide the benefit of version control, but the file can be hosted externally as well.

A job can be scheduled to pull down a file to update policy, and then add that job in a multi-step job that then installs policy right after the file is ingested on a set schedule.

Import External Policy to Management Center 
 

 

Policy download on Proxy

The CPL Local, Forward and Central files also allow you to import a file from a remote URL or local file option. The Central file can be used to automatically check for changes and import the file when it changes.

Configuring a Custom Central Policy File for Automatic Installation
It's possible to use custom categories defined by a local database which the Proxy can be configured to receive updates from.

Configure the Local Database

 

Standard CPL policy

The Large list of URLs could also be maintained in CPL which updating a bulk list of URLs would be more efficient than the VPM. This is still more of a manual option compared to the more recent policy imports, shared objects or API, but still a very simple and potential candidate here.
 

 

MC API

The Management Center API can be used to manage policy, including creating new objects, changing the policy content, and installing policy on a device. Tools and/or Scripts can be developed and configured to bulk update URLS with this method.