AI Assistant service memory usage is very high.

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention
• VMware Avi Load Balancer

Issue/Introduction

You are running SSP or Avi Operations and have encountered an alarm with any of the following description:

The memory usage of AI Assistant Platform {{ .ResourceID }}/{ pod } is currently { value }%, which is above the threshold value.

Example:

The memory usage of AI Assistant Platform intelligent-assist-tutor/intelligent-assist-tutor-7c874cfb56-lvrg4 is currently 93%, which is above the threshold value.

• The { pod } is dynamic and will be prefixed by the service name as shown above.
• The { value } will be dynamic and will represent the current memory usage of the { pod }.

Environment

vDefend SSP Version: 5.2 and later

Avi Operations: 5.2 and later

Cause

The current memory limits for {.ResourceID} is not enough to process the current load.

Resolution

Steps to resolve:

Try re-starting the deployment/statefulset. This should take care of any transient issues. This will impact any ongoing chat conversations.

1. Log into SSP-Installer using 'sysadmin' credentials

2. Get the pod name for the {{ .ResourceID }} , using 'k -n nsxi-platform get pods | grep {{ .ResourceID }}'

k -n nsxi-platform get pod <pod-name> -o jsonpath='{.metadata.ownerReferences[0].kind}'

3. Confirm the output is ReplicaSet and run:

k -n nsxi-platform rollout restart deployment {{ .ResourceID }} 

4. Wait for ~20 minutes and check if the alarm is auto-resolved. (Run 'k -n nsxi-platform get pods' - to check restarted pod are up)