AI Assistant LLM model compatibility issue detected.
search cancel

AI Assistant LLM model compatibility issue detected.

book

Article ID: 413766

calendar_today

Updated On:

Products

VMware vDefend Firewall VMware vDefend Firewall with Advanced Threat Prevention VMware Avi Load Balancer

Issue/Introduction

LLM Models and bundles are incompatible.

Environment

vDefend SSP >= 5.2

 

Cause

Each bundle when imported into AI Assistant will set the supported embedding and reasoning model to support user questions. Current SSP version do come with supported reasoning and embedding models. As the LLM Providers (e.g GCP) retire and shutdown the LLM models from the service, AI Assistant needs to migrate to use newer models. In such cases, SSP will update supported reasoning and embedding models. The supported models are available using /ai-assistants/llm-support?usage=AI_ASSISTANT_PLATFORM AI Assistant api if you are using your own cloud provider.

Possible causes:

  • There is expiry_time to the model used in AI Assistant.

  • Auto download bundle is disabled. In this case if SSP supported config is drifted, this alarm will trigger.

  • Auto download bundle is enabled. In this case, this alarm will trigger only when there are no new bundles available for current SSP version. In this case the only way to get new bundles with supported models is to upgrade SSP.

Resolution

To take latest model updated in AI Assistant switch to auto download mode. Goto System > Connection Mode & Bundles > Edit and select "Auto-Update from vTIS".

Check the incompatible_bundle_status metric value using metrics API -

POST /metrics/data

{  "resource_type": "SSPSERVICEAIASSISTANT",  "resource_ids": [    "*"  ],  "keys": [    "ai_assistant_platform.incompatible_bundle_status"  ]}

And use SSP AI Assistant /ai-assistants/llm-support?usage=AI_ASSISTANT_PLATFORM API to get supported embedding and IQ models.

If metric value shows:

  • MODELS_DEPRECATED

    This status means AI Assistant bundle need to switch to a valid, non-expired model combination.

    Follow below steps:

    • If you already enabled auto update, this alarm is transient and will go away when new bundle is auto updated depending on cause.

    • If you have manual update:
      1. Navigate to the vTIS portal, by pasting the following URL in a browser: https://portal.securityti.vmware.com
      2. Invoke SSP API, GET /ai-assistants/llm-support?usage=AI_ASSISTANT_PLATFORM API to check which embedding_model marked default=true. Manually download bundle of the default embedding from vTIS portal - Threat Intelligence Bundle > AI Assistant > Select SSP Version > select embedding model from above API. Download the bundle.

      3. Login to SSP and goto System → Platform & Features → AI assistant tile card (Firewall or Avi) → Edit Bundle management → on the AI Assistant Bundle page. Click on Manage Bundles and then click Upload Bundle button to upload new bundle.
  • SSP_UPGRADE_REQUIRED

    Upgrade the SSP is required because there is no support available for either expired or unavailable model. To support newer providers and models, AI Assistant will require backend change which will be available in newer SSP releases.

Few resolution options:

  • Enable auto download to import new bundle with supported embedding models.
  • If custom config is enabled, use newer reasoning or embedding model supported as indicated in the API
  • Upgrade SSP in case no new bundles are available for the embedding models.
Powered by Wolken Software