Avi Load Balancer Upgrade Failure from 22.1.6 to 30.2.3 with error: 'Cluster is missing required fields'

Products

• VMware Avi Load Balancer

Issue/Introduction

• When upgrading AVI controller, the controller upgrade fails.

• The UI shows upgrade has failed with no other error.

• The controller shell shows the error: UPGRADE_FSM_ABORTED

• 
• The following error in can be seen under the /var/lib/avi/log/process-supervisor.log
  
  

  • File "/opt/avi/python/lib/avi/util/protobuf.py", line 69, in protobuf_serializepb_str = pb.SerializeToString()google.protobuf.message.EncodeError: Message Cluster is missing required fields: nodes[0].interfaces[0].ip.mask,nodes[0].interfaces[0].ip.ip_addr.type,nodes[1].interfaces[0].ip.mask,nodes[1].interfaces[0].ip.ip_addr.type,nodes[2].interfaces[0].ip.mask,nodes[2].interfaces[0].ip.ip_addr.type^[[0m[2025-12-19 07:52:31,699] INFO [restore_datastore.handle:41] Restoring redis datastore objects from database....

Environment

VMware AVI Load Balancer

• 22.1.7

Cause

• The issue is caused due to a malformed/corrupted cluster.conf file under the /var/lib/avi/etc/ directory.
• The customer didn't configure the IPv6, though there were entries for it in the cluster.conf file with the addition of malformed %eth0.
  
  
• 

Resolution

• Remove the unnecessary IPv6 details.
• Get the correct vm_uuid. ssh to the controller leader node and cat /etc/vm_uuid
• Get the correct ipv4 mac_address
• Get the vm_mor ID from the vCenter. In vCenter, select the VM and take it from the browser.
  • 
• It's sample config of the /var/lib/avi/etc/cluster.conf file, It is expected to work.

name: cluster-0-1
nodes:
-   interfaces:
    -   gateway:
            addr: ##.##.##.##
            type: V4
        if_name: eth0
        ip:
            ip_addr: 
                addr: ##.##.##.##
                type: V4
            mask: 23
        labels:
        - MGMT
        - SE_SECURE_CHANNEL
        - HSM
        mac_address: ##:##:##:##:##:##
        mode: STATIC
    ip:
        addr: ##.##.##.##
        type: V4
    name: ##.##.##.##
    vm_hostname: node3.controller.local
    vm_mor: ''
    vm_uuid: ####-####-####-####-####
-   interfaces:
    -   gateway:
            addr: ##.##.##.##
            type: V4
        if_name: eth0
        ip:
            ip_addr:
                addr: ##.##.##.##
                type: V4
            mask: 23
        labels:
        - MGMT
        - SE_SECURE_CHANNEL
        - HSM
        mac_address: ##:##:##:##:##:##
        mode: STATIC
    ip:
        addr: ##.##.##.##
        type: V4
    name: ##.##.##.##
    vm_hostname: node2.controller.local
    vm_mor: ''
    vm_uuid: ####-####-####-####-####
-   interfaces:
    -   gateway:
            addr: ##.##.##.##
            type: V4
        if_name: eth0
        ip:
            ip_addr:
                addr: ##.##.##.##
                type: V4
            mask: 23
        labels:
        - MGMT
        - SE_SECURE_CHANNEL
        - HSM
        mac_address: ##:##:##:##:##:##
        mode: STATIC
    ip:
        addr: ##.##.##.##
        type: V4
    name: ##.##.##.##
    vm_hostname: node1.controller.local
    vm_mor: ''
    vm_uuid: ####-####-####-####-####
tenant_uuid: admin
uuid: cluster-####-####-####-####-####
virtual_ip:
    addr: ##.##.##.##
    type: V4