Authentication Error When Performing a Scale/Reconfigure Network Function Operation

Products

• VMware Telco Cloud Automation

Issue/Introduction

Symptoms:

Network function scale or reconfigure operations fail with an authentication error message after a CNF upgrade operation.

Examples:
{"Operation":"Upgrade","level":"error","msg":"error getting chart authentication error: invalid repository username and/or password","time":"2023-03-27T09:23:03Z"}

or 
2023-03-27 09:23:06.005 UTC [NfvCnfLcmService_SvcThread-3819, Ent: HybridityAdmin, Usr: HybridityAdmin, , TxId: ########-####-####-####-############] ERROR com.vmware.hybridity.nfvm.JobUtils- CN_CHART_RECONFIGURE failed, errorCode:null. stacktrace:null, response: {"errors":[{"internalMessage":"Helm Upgrade Failure: Helm API failed: authentication error: invalid repository username and\/or password: authentication error: invalid repository username and\/or password","errorCode":"E_CNF_HELM_RECONFIGURATION"}]}

 

Environment

2.0.x

Cause

TCA loses the Harbor credentials stored in the TCA database after a CNF upgrade operation is performed. 

Resolution

Resolved in 2.1+

Workaround:

Manually add the Harbor credentials into the TCA database.

Note: Make a backup of the database by generating the TCA log bundle as documented in the TCA Deployment Guide. Ensure the option to include the MondoDB dump is selected. 

1. SSH into the TCA Manager appliance using the admin account.
2. Switch user to root. 
   su
3. Access the database with the following command:
   mongo hybridity
4. Gather the nfInstanceId from the error message shown in the TCA UI.
   In this example, the nfInstanceId is 8f6a9b3f-d5e1-4de1-975b-9682389a7548.
5. Gather the entityId by running the following query in MongoDB, use the nfInstanceId obtained in the previous step. 
   
   In this example, the entityId is highlighted in red for reference.:
   db.Vnf.find({"id":" 8f6a9b3f-d5e1-4de1-975b-9682389a7548"}).pretty()
   
   Sample output:

      {
       "_id" : ObjectId("6414b88c7db8c2139155135f"),
       "id" : "8f6a9b3f-d5e1-4de1-975b-9682389a7548",
       "vnfInstanceName" : "test",
       "vnfInstanceDescription" : ""
       "vnfdId" : "nfd_d3fcc95b-877e-4a0c-ba05-01fedbdd6eff",
       "vnfPkgId" : "47a8691a-030a-4fae-bfad-7e1d25d1a55b",
       "vnfCatalogName" : " test",
       "vnfProvider" : "company_name",
       "vnfProductName" : "product_name",
       "vnfSoftwareVersion" : "software_version",
       "vnfdVersion" : "descriptor_version",
       "onboardedVnfPkgInfoId" : "47a8691a-030a-4fae-bfad-7e1d25d1a55b",
       "instantiationState" : "INSTANTIATED",
       "managedBy" : {
              "extensionSubtype" : "VMWARE-TELCO-GVNFM",
              "extensionName" : "VMware"
       },
       "nfType" : "CNF",
       "lastUpdated" : ISODate("2023-03-17T21:29:34.984Z"),
       "lastUpdateEnterprise" : "HybridityAdmin",
       "lastUpdateOrganization" : "HybridityAdmin",
       "lastUpdateUser" : "HybridityAdmin",
       "creationDate" : ISODate("2023-03-17T18:59:24.310Z"),
       "creationEnterprise" : "HybridityAdmin",
       "creationOrganization" : "HybridityAdmin",
       "creationUser" : [e-mail],
       "isDeleted" : false,
       "vimConnectionInfo" : {
              "id" : "vmware_A3DC59AEA7464F67BF59AF9E6D47012E",
              "vimName" : "workload"
       },
       "lcmOperation" : "UPGRADE","lcmOperationState" : "FAILED","rowType" : "CNF",
       "instantiatedNfInfo" : {
             "helm1-8f6a9b3f-d5e1-4de1-975b-9682389a7548" : {
                  "dispatchType" : "tosca.nodes.nfv.Vdu.Compute.Helm.helm1",
                  "deploymentName" : "mysql-163-bb91a-tolgp",
                  "vduId" : "helm1",
                  "entityId" : "helm1-8f6a9b3f-d5e1-4de1-975b-9682389a7548",
                  "namespace" : "some-namespace",
                  "helmVersion" : "v3",
                  "chartName" : "mysql",
                  "chartVersion" : "1.6.3",
                  "repoUrl" : http://ipaddress/chartrepo/testing-cnf-def,
                  "extensionId" : "",
                  "username" : "admin",
                  "password" : "somePassword"
                  },
              "helm2-8f6a9b3f-d5e1-4de1-975b-9682389a7548" : {
                  "dispatchType" : "tosca.nodes.nfv.Vdu.Compute.Helm.helm2",
                  "deploymentName" : "nginx-ingre-bb91a-ac0bf",
                  "vduId" : "helm2",
                  "entityId" : "helm2-8f6a9b3f-d5e1-4de1-975b-9682389a7548",
                  "namespace" : "some-namespace",
                  "helmVersion" : "v3",
                  "chartName" : "nginx-ingress",
                  "chartVersion" : "1.41.2",
                  "repoUrl" : http://ipaddress/chartrepo/testing-cnf-def,
                  "extensionId" : "",
                   "username" : "admin",
                   "password" : "somePassword"
                }
          }

6. Using the values collected previously, run the following command to re-add the credentials into the database.

db.Vnf.updateMany({"id":"nfInstanceId"},{"$set":{"instantiatedNfInfo.entityId1.username":"harborUserName”,"instantiatedNfInfo.entityId1.password":"harborPassword" }})

Note: Replace the following placeholder values with actual values from your environment:
nfInstanceId
entityId1
harborUserName
harborPassword
 

Additional Information