Is there a built-in brute-force protection for passwords in VIP Enterprise Gateway console (for example, a lockout rule for failed attempts)? Is there other protection method?
There is no 'built-in protection' against brute-force attacks per se for VIP Enterprise Gateway. The workaround is to implement Console Authentication (User Store > Console Authentication) by way of using LDAP/AD's own password policies (for instance, locking an account after 5 incorrect password attempts).