Virtual Service deployment fails with the following the error: "Service Engine Avi-se-xxxxx in the Service Engine Group <seg-name> is unavailable - SE reached max[max-value] consumers"

Products

• VMware Avi Load Balancer

Issue/Introduction

Virtual Service deployment fails with the following the error: "Service Engine Avi-se-xxxxx in the Service Engine Group <seg-name> is unavailable - SE reached max[max-value]" consumers

For Example:

Environment

vCenter/vSphere

Cause

For Example:

In the res_mgr.INFO file on the leader node in the controller cluster under /var/lib/avi/log/glog, we see the following error:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

tenant: I1114 20:09:32.104776  5528 rm.cc:4109] T[a2d3e216] F[FindRestrictedSeListToUse] [HTTP_VS:virtualservice-434a1f:1] found legacy active-standby primary SE[Avi-se-xxxxx:se-49f2bd] for placement
I1114 20:09:32.104789  5528 rm.cc:4368] T[a2d3e216] F[FindSeResourceInternal] [HTTP_VS:virtualservice-434a1f:1] regular consumer load_needed[1]
I1114 20:09:32.104810  5528 rm.cc:3367] T[a2d3e216] F[RunPlacementAlgoDistributed] [HTTP_VS:virtualservice-434a1f:1] creation[disabled] vip_shared[0] max_se_load[0] max_se_inp_load[0] max_vs_per_se[1] load_needed[1]
I1114 20:09:32.104820  5528 rm.cc:4905] T[a2d3e216] F[FindSeResource] [HTTP_VS:virtualservice-434a1f:1] PLACEMENT_INELIGIBLE_ACTIVE_STANDBY_CONSTRAINTS: Avi-se-mztkm, PLACEMENT_INELIGIBLE_SE_MAX_VS_PER_SE_LIMIT: Avi-se-xxxxx, Avi-se-xxxxx,
I1114 20:09:32.161391  5528 rm_svc_impl.cc:1659] T[] F[ResMgrSCVsStateInfoUpdCb] [HTTP_VS:virtualservice-434a1f:1] create/update notification:
uuid: "scvsstateinfo-8fbcac89-e8f9-483b-96b8-2aa338099cd6"
state: OPER_RESOURCES, reason: "Service Engine Avi-se-xxxxx in the Service Engine Group Active_xxxxx is unavailable - SE reached max[1] consumers"

If we check the SE group configuration, we have an Active/Standby SE group with Virtual Services per Service Engine set to 1[Max value]

Resolution

Increase the value of 'Maximum Number of Virtual Services per Service Engine' for the problematic Active/Standby Service Engine group on which the VS is supposed to be placed to accommodate the new VS.

Note: Increasing the VSes per SE will lead to an increased resource consumption on the SE. 

Additional Information

Note:

1. Timestamps may vary in the customer environment. 
2. The res_mgr.INFO file path is as follows: admin@xx-xx-xx-xx:/var/lib/avi/log/glog$ The IP should be of the leader node in the controller cluster.
3.The max number of VSes allowed per SE is 1000. If the number of VSes goes beyond 1000, a new SE will need to be created to host the VS.