AI Assistant cannot connect to LLM endpoint.

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention
• VMware Avi Load Balancer

Issue/Introduction

AI Assistant has lost connectivity to LLM endpoint.

Environment

vDefend SSP Version: 5.2 and later

Avi Operations: 5.2 and later

Cause

There can be few reasons the AI Assistant can not connect to LLM endpoint.

1. Network/Proxy not allowing connectivity.
2. Invalid LLM Key which fails to connect cloud provider.
3. Check for vTIS connectivity: Review SSP system dashboard in UI > Home > Open Alarms to make sure there are no active vTIS (Threat Intelligence Service) connectivity alarms blocking the service.

Resolution

• Wait for Automatic Recovery
  
  
  • In many cases, once the root cause is resolved, the connection should automatically recover.
    
    
  • It is recommended to wait 5-10 minutes to allow the system to attempt an automatic reconnection before proceeding with any of the below steps.
    
    
• Ensure that the ports required for AI Assistant is allowed in the network. Use Ports and Protocols and filter "AI Assistant" in "Purpose" column to find all required port-protocol.
  
  
• Verify SSP proxy settings that allows your provider endpoints access via proxy.

  • If applicable, ensure that your proxy settings are correctly configured. Please follow these steps:

    • Verify Connectivity to Proxy Server

      • Contact your network administrator to confirm:

        • The firewall allows traffic between the SSP subnet(Node IP Pool) and the proxy server.
        • The proxy server has internet access.
          
          
      • Verify Proxy Server Configuration
        
        
        • Contact the proxy server administrator to confirm:
          
          
          • The proxy server is operational.
            
            
      • • The correct configuration settings, including:
          
          
          • Proxy scheme (HTTP/HTTPS), host address, and port number.
          • Proxy credentials (username and password).
          • If necessary, import the updated server certificate.
            
            
      • Update Proxy Configuration
        
        
        • If any updates are required, go to System → Server Configurations on the SSP Platform and edit the Internet Proxy Server settings to reflect the correct information.
          
          
• In case of Broadcom account, ensure that connectivity to the vTIS endpoint URL https://api.prod.nsxti.vmware.com is not blocked anywhere in the upstream network. Review any recent changes to firewalls, DNS settings, or network configurations that might be restricting access. If any changes were made, revert them to allow access to the above vTIS URL. 
  
  
  • Check if there is existing vTIS connectivity alarm and follow instructions vDefend SSP Alarm: VTICS/VTIS Service is unreachable
    
    
• Please ensure that the BYO LLM key is valid and not expired.