Issues with handling object names with leading and trailing whitespace characters

Products

• VMware Avi Load Balancer

Issue/Introduction

• Upgrade may fail with error: "Unable to complete task MigrateConfig".
• SE creation may fail after the first one is created in the SE Group with the error "ServerFaultCode: The name <SE-Folder-Name> already exists."
• AKO may fail to attach the correct SSL certificate to the VS with the error "SSLKeyAndCertificate matching query does not exist."

Environment

• SE creation failures are seen only for NSX-T clouds. Avi versions 22.1.7+, 30.2.2+ and 31.1.1+ are susceptible to this.
• The other issues may be observed on ANY Avi deployment.

Cause

• The issues of upgrade failure and AKO not linking the correct certificates to the VSs are caused because of SSL certificate objects having a leading whitespace in their name.
• Because of how this is handled internally, the controller incorrectly reports that the SSL certificate does not exist causing the failures. 
• The SE creation error is seen after the first SE has been created in the SE Group. All SE creates post this fail with the error mentioned above.
• This is because of the following sequence of events:
  • The vCenter Folder field in NSX-T SE Groups is a text box starting with the versions mentioned above.
  • When this name is configured with a leading whitespace character, the controller queries to check if the folder exists in vCenter.
  • vCenter does NOT allow leading whitespaces in names, hence, the response to the above query is always to be negative.
  • For the first SE, the controller tries to create the folder with the leading whitespace in the name on vCenter. 
  • vCenter silently trims the whitespace and creates the folder. The SE is placed in this folder. 
  • For ALL subsequent SE creates, the controller will try to create a folder with the leading whitespace in its name and vCenter will respond with "ServerFaultCode: The name <SE-Folder-Name> already exists."

Resolution

• Our teams are working on identifying the best solution for these issues.
• The current recommendation is to avoid adding leading/trailing whitespaces to object names on Avi.
• To verify if SSL Certificates or vCenter Folder names have whitespace characters, you can use the command below:
  • For vCenter Folders:
    

    jq '.ServiceEngineGroup[] | .vcenter_folder + " " + .name + " " + .cloud_ref' <avi_config_file> | grep '%20'

  • If you see "%20" at the beginning or ending of the first column output, then you could run into some issue. Please modify those SE Group configurations accordingly.

  • For SSL certificates:

    jq '.SSLKeyAndCertificate[] | .name' <avi_config_file> | grep '%20'

  • If you see "%20" at the beginning or ending of the name, then you could run into some issue. Please modify those names accordingly.