Error: The resolved IP address from the FQDN is already in use during Identity Broker deployment - VCF 9.1
search cancel

Error: The resolved IP address from the FQDN is already in use during Identity Broker deployment - VCF 9.1

book

Article ID: 443128

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer VCF Operations

Issue/Introduction

  • During an upgrade from VCF 9.0.x to 9.1, the "Deploy VCF Management Components" workflow may fail at the "Install Identity Broker using Fleet Lifecycle" subtask
  • Below error is reported on the installer 
    Install Identity Broker using Fleet Lifecycle
    Failed
    Description    Install Identity Broker using Fleet Lifecycle
    Progress Messages    The Fleet lifecycle 'VSP component installation' task with ID <subtask ID> failed. Internal errors: The resolved IP address <IP address> from the FQDN <Identity broker FQDN> is already in use.
    Error
    
    Message: The Fleet lifecycle 'VSP component installation' task with ID <subtask ID> failed. Internal errors: The resolved IP address <IP address> from the FQDN <Identity broker FQDN> is already in use.
  • /var/log/vmware/vcf/domainmanager/domainmanager.log on SDDC Manager 
    ERROR [vcf_dm,<OP ID>,ad1a] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-31]  [IFCN5A] PUBLIC_FLEET_LCM_TASK_FAILED The Fleet lifecycle {0} task with ID <ID> failed. Internal errors: The resolved IP address <IP Address> from the FQDN <Identity broker FQDN> is already in use.
    
    com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: The Fleet lifecycle {0} task with ID <ID> failed. Internal errors: The resolved IP address <IP Address> from the FQDN <Identity broker FQDN> is already in use.
    
    DEBUG [vcf_dm,<OP ID>,28fc] [c.v.e.s.o.c.c.ContractParamBuilder,dm-exec-47]  Contract task InstallVspComponentFleetLcmAction input: {"_executionUuid":"<execution_ID>","componentSpec":{"deploymentType":"VspComponentSpec","sddcLcmId":"<LCM OP ID>","componentType":"VIDB","version":"9.1.0.0.25368698","repository":{"downloadUrl":"https://<Identity broker FQDN>/vmware/vcf/bundle/<bundle ID>/<bundle ID>/vidb-9.1.0.0.25368698.tgz"},"fqdn":"<Identity broker FQDN>","componentNamespace":"vidb-external","configSpec":{"size":"small","ipAddresses":[]}}}

Environment

VMware SDDC Manager / VCF Installer 

Cause

  • The Identity Broker deployment requires an unique FQDN and a fresh IP address. If the FQDN provided in the wizard resolves to an existing IP (such as the vCenter FQDN), the Fleet Lifecycle manager fails validation.

Resolution

  1. In order to resolve the issue, the existing configuration has to be re-initiated post updating the FQDN for VIDB (Identity Broker)
  2. Log in to SDDC Manager machine using vcf user and switch to root user using su
  3. Add vcf.vault.http-access=true under application-prod.properties
    echo vcf.vault.http-access=true >> /etc/vmware/vcf/domainmanager/application.properties && systemctl restart domainmanager
    

    Note: After the workflow completes successfully, delete the property vcf.vault.http-access=true from /etc/vmware/vcf/domainmanager/application-prod.properties

  4. Execute the below command to export the json for the current configuration
    curl -X GET http://localhost/domainmanager/internal/vault/<execution_uuid> | json_pp > /tmp/workflow.json

Note: execution_uuid is captured from domainmanager.log

Sample:- input: {"_executionUuid":"<execution_ID>"

  1. Edit /tmp/workflow.json and replace instances of the incorrect Identity Broker FQDN with a new & unique FQDN

    The specific sections where the FQDN needs to be updated:

    DeployOrJoinVcfManagementComponents____##DeployVidbFleetLcm##__componentSpec
    DeployOrJoinVcfManagementComponents____##__vcEndpoint
    vidbSpec
    vidb
  2. Update the vault with the modified JSON 
    curl -H 'Content-Type:text/plain' -X PUT http://localhost/domainmanager/internal/vault/<execution_uuid> -d @/tmp/workflow.json
  3. Restart the "Deploy VCF Management Components" Task

    1. In the VCF Operations UI browse to Build > Lifecycle > Tasks
    2. Locate the most recent Failed status Task for "Install components: Identity broker"
    3. Click on the failed task and copy the Task ID (UUID format) from the task details
    4. SSH to the SDDC Manager appliance as the vcf user and switch to root user
    5. Open the application-prod.properties file for editing with the following command

      vi /etc/vmware/vcf/domainmanager/application-prod.properties
    6. Add or update the following entry, replacing <Task_ID from step c> taken from Step 3 above and save the file 
      fleetlcm.task.id.to.recreate=<Task_ID from step c>
    7. Restart domain manager service on the SDDC manager appliance using the following command 
      systemctl restart domainmanager
    8. Return to the VCF OPs UI, navigate to Build > Tasks, locate the "Deploy VCF Management Components" task and click Restart Task