Network function reconfigure operation fails at 80 %

Products

• VMware Telco Cloud Automation

Issue/Introduction

Network function (NF) reconfigure operation fails at 80 %

- Scale/reconfigure operations fail with an authentication error message after a CNF upgrade operation.
-  Helm Upgrade Failure: Helm API failed: authentication error



- Scale Operation failed with error. Auto rollback successful.

In the status detail you notice the following Error:



At TCA-CP app.log error snippet:
 

2023-03-27 09:23:06.001 UTC [NfvCnfLcmService_SvcThread-3819, Ent: HybridityAdmin, Usr: HybridityAdmin, , TxId: ########-####-####-####-############] INFO c.v.h.c.workflows.CnfReconfigureJob- Fetching the status of dispatched orchestration jobs


2023-03-27 09:23:06.005 UTC [NfvCnfLcmService_SvcThread-3819, Ent: HybridityAdmin, Usr: HybridityAdmin, , TxId:########-####-####-####-############] ERROR com.vmware.hybridity.nfvm.JobUtils- CN_CHART_RECONFIGURE failed, errorCode:null. stacktrace:null, response: {"errors":[{"internalMessage":"Helm Upgrade Failure: Helm API failed: authentication error: invalid repository username and\/or password: authentication error: invalid repository username and\/or password","errorCode":"E_CNF_HELM_RECONFIGURATION"}]}

 

 

Environment

2.0.1

Cause

TCA loses the Harbor credentials stored in the TCA database after a CNF upgrade operation is performed.

Resolution

Resolved in TCA 2.1+

Workaround:

Add the Harbor credentials manually into the TCA Manager database or upgrade TCA to the latest version.

Procedure

• SSH into the TCA Manager appliance using the admin account.
• Backup the DB before update/edit

Backup:
mongodump --archive=/tmp/mongoExport/mongo_hybridity.gz --gzip --db hybridity --excludeCollection=audittrail

TO RESTORE (IF APPLICABLE) 
mongorestore --gzip --archive=mongo_hybridity.gz --db hybridity

• Switch user to root user
  su -
• Edit the database:
  NOTE: Update harbor username and password for multiple NFs in TCA-M. You will need to replace <<entityId1>> with the actual  entityIds from the DB and run the command for that particular nfInstanceId. Please replace <<harborUsername>> and <<harborPassword>> with the actual values.

mongo hybridity
db.Vnf.updateMany({"id":"<<nfInstanceId>>"},{"$set":{"instantiatedNfInfo.<<entityId1>>.username":"{{harborUsername}}","instantiatedNfInfo.<<entityId1>>.password":"<<harborPassword>>" }})

Example : 
db.Vnf.updateMany({"id":"f99dfd9b-a8f3-44f3-a9bf-35e46196ce88"},{"$set":{"instantiatedNfInfo.log-distributor-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.log-distributor-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.reprocessor-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.reprocessor-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.dr-redis-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.dr-redis-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.dr-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.dr-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.output-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.output-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.dr-cmdb-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.dr-cmdb-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.configuration-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.configuration-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.mapper-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.mapper-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.input-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.input-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword","instantiatedNfInfo.kafka-output-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.username":"yourusername","instantiatedNfInfo.kafka-output-stream-f99dfd9b-a8f3-44f3-a9bf-35e46196ce88.password":"samplepassword"}})

Additional Information

Unable to Scale/Reconfigure after an CNF upgrade operation.