Authentication Error When Performing a Scale/Reconfigure Network Function Operation in Telco Cloud Automation (TCA) 2.0.X

Products

VMware VMware Telco Cloud Automation

Issue/Introduction

Symptoms:

Network function scale or reconfigure operations fail with an authentication error message after a CNF upgrade operation.

Examples:
{"Operation":"Upgrade","level":"error","msg":"error getting chart authentication error: invalid repository username and/or password","time":"2023-03-27T09:23:03Z"}

or
2023-03-27 09:23:06.005 UTC [NfvCnfLcmService_SvcThread-3819, Ent: HybridityAdmin, Usr: HybridityAdmin, , TxId: 0c5a54fc-0cea-47db-b3a4-65b82d05f0c9] ERROR com.vmware.hybridity.nfvm.JobUtils- CN_CHART_RECONFIGURE failed, errorCode:null. stacktrace:null, response: {"errors":[{"internalMessage":"Helm Upgrade Failure: Helm API failed: authentication error: invalid repository username and\/or password: authentication error: invalid repository username and\/or password","errorCode":"E_CNF_HELM_RECONFIGURATION"}]}

Environment

VMware Telco Cloud Automation 2.0
VMware Telco Cloud Automation 2.0.1

Cause

TCA loses the Harbor credentials stored in the TCA database after a CNF upgrade operation is performed.

Resolution

This issue has been resolved in VMware Telco Cloud Automation 2.1 or later TCA versions.

Workaround:

Manually add the Harbor credentials into the TCA database.

Note: Make a backup of the database by generating the TCA log bundle as documented in the TCA Deployment Guide. Ensure the option to include the MondoDB dump is selected.

SSH into the TCA Manager appliance using the admin account.
Switch user to root.
su
Access the database with the following command:
mongo hybridity
Gather the nfInstanceId from the error message shown in the TCA UI.
In this example, the nfInstanceId is 8f6a9b3f-d5e1-4de1-975b-9682389a7548.
Gather the entityId by running the following query in MongoDB, use the nfInstanceId obtained in the previous step.

In this example, the entityId is highlighted in red for reference.:
db.Vnf.find({"id":" 8f6a9b3f-d5e1-4de1-975b-9682389a7548"}).pretty()

Sample output:

      {
       "_id" : ObjectId("6414b88c7db8c2139155135f"),
       "id" : "8f6a9b3f-d5e1-4de1-975b-9682389a7548",
       "vnfInstanceName" : "test",
       "vnfInstanceDescription" : ""
       "vnfdId" : "nfd_d3fcc95b-877e-4a0c-ba05-01fedbdd6eff",
       "vnfPkgId" : "47a8691a-030a-4fae-bfad-7e1d25d1a55b",
       "vnfCatalogName" : " test",
       "vnfProvider" : "company_name",
       "vnfProductName" : "product_name",
       "vnfSoftwareVersion" : "software_version",
       "vnfdVersion" : "descriptor_version",
       "onboardedVnfPkgInfoId" : "47a8691a-030a-4fae-bfad-7e1d25d1a55b",
       "instantiationState" : "INSTANTIATED",
       "managedBy" : {
              "extensionSubtype" : "VMWARE-TELCO-GVNFM",
              "extensionName" : "VMware"
       },
       "nfType" : "CNF",
       "lastUpdated" : ISODate("2023-03-17T21:29:34.984Z"),
       "lastUpdateEnterprise" : "HybridityAdmin",
       "lastUpdateOrganization" : "HybridityAdmin",
       "lastUpdateUser" : "HybridityAdmin",
       "creationDate" : ISODate("2023-03-17T18:59:24.310Z"),
       "creationEnterprise" : "HybridityAdmin",
       "creationOrganization" : "HybridityAdmin",
       "creationUser" : [email protected],
       "isDeleted" : false,
       "vimConnectionInfo" : {
              "id" : "vmware_A3DC59AEA7464F67BF59AF9E6D47012E",
            "vimName" : "workload"
       },
       "lcmOperation" : "UPGRADE","lcmOperationState" : "FAILED","rowType" : "CNF",
       "instantiatedNfInfo" : {
             "helm1-8f6a9b3f-d5e1-4de1-975b-9682389a7548" : {
                  "dispatchType" : "tosca.nodes.nfv.Vdu.Compute.Helm.helm1",
                  "deploymentName" : "mysql-163-bb91a-tolgp",
                  "vduId" : "helm1",
                  "entityId" : "helm1-8f6a9b3f-d5e1-4de1-975b-9682389a7548",
                  "namespace" : "some-namespace",
                  "helmVersion" : "v3",
                  "chartName" : "mysql",
                  "chartVersion" : "1.6.3",
                  "repoUrl" : http://ipaddress/chartrepo/testing-cnf-def,
                  "extensionId" : "",
                  "username" : "admin",
                  "password" : "somePassword"
                  },
              "helm2-8f6a9b3f-d5e1-4de1-975b-9682389a7548" : {
                  "dispatchType" : "tosca.nodes.nfv.Vdu.Compute.Helm.helm2",
                  "deploymentName" : "nginx-ingre-bb91a-ac0bf",
                  "vduId" : "helm2",
                  "entityId" : "helm2-8f6a9b3f-d5e1-4de1-975b-9682389a7548",
                  "namespace" : "some-namespace",
                  "helmVersion" : "v3",
                  "chartName" : "nginx-ingress",
                  "chartVersion" : "1.41.2",
                  "repoUrl" : http://ipaddress/chartrepo/testing-cnf-def,
                  "extensionId" : "",
                   "username" : "admin",
                   "password" : "somePassword"
                }
          }

Using the values collected previously, run the following command to re-add the credentials into the database.

db.Vnf.updateMany({"id":"nfInstanceId"},{"$set":{"instantiatedNfInfo.entityId1.username":"harborUserName”,"instantiatedNfInfo.entityId1.password":"harborPassword" }})

Note: Replace the following placeholder values with actual values from your environment:
nfInstanceId
entityId1
harborUserName
harborPassword

Additional Information

Impact/Risks:
Impacts VMware Telco Cloud Automation 2.0, 2.0.1.