Wildcards in Proxy Exclude not Enforced Correctly. "ProxyHTTP: java.io.IOException: proxy error: Forbidden"

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:
VM deployment fails on an environment with an external proxy, configured to block traffic from addresses entered as wildcard IPs in the proxy-exclude list of the VA.

Environment

VMware Aria Automation 8.x

Cause

Due to misconfiguration in the internal k8s squid proxy server, proxy-exclude addresses added as a partial IP address (eg. "10." or "192.168.") will not be respected by the proxy.
Wildcard domain addresses (eg. .abc.com) in the proxy-exclude list are not impacted by this.

Resolution

The issue will be resolved in VMware Aria Automation 8.14.0.

Workaround:

Prerequisites:

• Please take simultaneous non-memory snapshots of each virtual appliance(s) in the cluster.
• You have access to root user and password
• You have SSH or console access to each virtual appliance.

Installation Procedure:
1. SSH / PuTTy into one Aria Automation virtual appliance in the cluster
2. Run the following command:

base64 -d <<< '/Td6WFoAAATm1rRGAgAhARwAAAAQz1jM4AjwA2JdABGIQkY99BhqpmeveqFJqtoo9sIUIWHt3roBcfzI8xCoFUaDbKGbVntx2tZ9s/IEM/4WXqUgB7/FzMDkUwGos6ea+fDOPWjoh2Xqzn1YttjSGI/9MqFnREwnk8HpTddeBZQW+4tG/Sy1m+Hz076ygfLkoWnXXa2nhpOJshb06+jrshEkXaeSvZYi8VH8dk9SHRXKM9JUw127OkDPeefALVVK7H3W0ZKAPHwen1wLZRXsA5V7DqrwiyLx4oLlZtzc3PvBKRiLQQdlgPAIS28EgUGBgEroXHHFmgZ1J7LJ8+/LrzV/WqXSQ/G+N7YUyV4lHQXZfmEg9sJYoJzOCXlsNEoXjZrorp8Cv47nHkS7iSgOYMDBqKF02zQxAhxfy/GwARSGAIuzg+G2/c+Vxu97+ECxKz2apX4JK1lwmnHz76bv/8XU62lbcJqUfwGBc33IuaZMYHuqtQ+ZdY445ATG1lab88KQVqiX2nvQ9PgUm7zrpX0Aw0r/NeoHTnCW+3xWc8JeEmeP6CkTAwfBdiOgFsam9F8Etl3g9gNquO40DbkcvQirFU4Mp5QsVy2X+Ov1QuGU/AvXODI9Z9SJnlCY5U5EO8VGffCpxW/fuB5FMBeQrHcxU41iqDlfceel1VEEZYzFX9qOQ8cDF8zoJ2EPNHN1xAU4p3ZUN5tm/MOkTjwtStxCqSWYeBhAZEPEGqYcgHQdWOwVI1JB9qFcNJlnanuEp/vyOnEem5O4Go/uIpCAMFyGGoUIyTYHvvRt7GOufI2FLE0aMu3g7Dqog3t3wO7FukdyaHzXnsi318ye1IFManFSlOibiEQG+AJG6Al9rHNO9lF5jEjoUNsrHq5dhTcLLN0an6N4OwvgnQPbDbUME5qA+9MICwT7g4KLHd56sBnVn7HSwOI9E6xlBAUos5gH1N+72FxKV3XZY0DqA04wnYaBOjhhviQdVMKjmsWFCxJAjC1dv9ZcHa5Zt0thf6027BnfqLfdyGuUJCW1rkDmXxygDheHYZn7nIO4T4TMcov5e8EW5HUvQ0Syjai4pSsRa1BsZ5LB2pxK17p5FslxVYAH8nlk8EuXTWnWLYoV29qfw8qnTYViBnoX1piKKp80RBRBNk/Z442ijY38ujQbfP1vhSxE8SRHisypnoH8GWUKAAAASycfkee3kSwAAf4G8REAAJHhs0CxxGf7AgAAAAAEWVo=' | xz -d | bash -

Note: Once the patch is installed successfully, the backup files that were stored in /data/patch-vrae-53391/ may be removed to free space.

3. Confirm the patch has been applied successfully by running the following command on each node:

vracli proxy show | jq '.["internal.proxy.config"]' | tr -d '\n' |  sed 's/\\n/\n/g'

There should be 2 separate ACLs defined - proxy-exclude-ip and proxy-exclude-domain, containg ip/subnet mask and domain names respectively
Example:
acl proxy-exclude-domain dstdomain .local
acl proxy-exclude-domain dstdomain localhost
acl proxy-exclude-ip dst 10.0.0.0/8
acl proxy-exclude-ip dst 192.168.0.0/16
acl proxy-exclude-ip dst 127.0.0.1/32
acl proxy-exclude-domain dstdomain kubernetes
acl proxy-exclude-domain dstdomain .eng.vmware.com
always_direct allow proxy-exclude-ip
always_direct allow proxy-exclude-domain

Revert/Uninstall the patch:
The original backed-up file is saved as '/data/patch-vrae-53391/proxy_config.py.bak'. If needed, it can be restored to its original contents by running:

vracli cluster exec -- bash -c "cp /data/patch-vrae-53391/proxy_config.py.bak /opt/python-modules/vracli/proxy_config.py; rm -f /data/patch-vrae-53391/proxy_config.py.bak"

Additional Information

Note: If you upgrade to any version before that, this patch will be undone and it will need to be re-applied after the upgrade.

Impact/Risks:
In an environment with an external proxy, and proxy-exclude list containing wildcard IP addresses (eg. "10." or "192.168.") services will fail to come up and consequentially deployment will fail as well.