The ProxySG appliane returns Format Error when a specific application send DNS queries and you are using the ProxySG appliance as a DNS proxy.
SGOS prior 6.7.4.1 does not support EDNS queries.
SGOS 6.7.4.1 support EDNS, but default configlation of EDNS is disabled.
Per RFC2671, ProxySG returns the Format Error if it receives EDNS queries.
SGOS prior 6.7.4.1:
Consider changing the configuration of the application, for example, if there is a problem with the the application traffic.
SGOS 6.7.4.1 or higher:
Use the following CLI to enable EDNS:
#(config)dns edns enable