The Visual Policy Manager's Web Content Layer does not give you an option of applying a source IP for a given action. The workaround is to do this using the CPL under the Local Policy.
For example, if you want to bypass ICAP Respond scanning:
Go to Management Console -> Configuration -> Policy -> Policy Files -> Policy Files tab -> Install Local File from: Text Editor -> Click on the Install and add the policy below:
<Cache>
client.address=172.16.172.72 response.icap_service(no)