Network Functions get stuck when instantiating a CNF instead of timing out.
Article ID: 325404
Updated On:
Products
VMware
VMware Telco Cloud Automation
Issue/Introduction
The purpose of this patch is to fix the memory leak identified and introduce a timeout for operations that are in a stuck state and limit the number of helm-service threads that are active on the Telco Cloud Automation Control Plane (TCA-CP)) at any given moment.
Symptoms:
Network Functions get stuck when instantiating a CNF instead of timing out.
Symptoms:
Network Functions get stuck when instantiating a CNF instead of timing out.
Environment
VMware Telco Cloud Automation 2.0.1
VMware Telco Cloud Automation 2.0
VMware Telco Cloud Automation 2.0
Cause
In certain scenarios the memory footprint of the helm-service, both RAM and swap, will drastically increase. This will ultimately lead to an out of memory condition.
Resolution
Resolved in Telco Cloud Automation 2.1.
Workaround:
1. Download the appropriate helm-service binary for your version of TCA:
helm-service_patch_2.0.0.1
helm-service_patch_2.0.1
2. SSH into the TCA-CP virtual machine.
3. Stop the helm-service
systemctl stop helm-service
4. Rename the binary to helm-service and copy the new binary to the following folder:
/opt/vmware/bin/
5. Add executable permissions to the binary
chmod 775 /opt/vmware/bin/helm-service
6. Start the helm-service
systemctl start helm-service
New environment variables:
To apply new environment variables:
Add the environment variables by modifying the systemd environment file located in the following folder of the TCA-CP
/etc/hybridity/helm-service.conf
Workaround:
1. Download the appropriate helm-service binary for your version of TCA:
helm-service_patch_2.0.0.1
helm-service_patch_2.0.1
2. SSH into the TCA-CP virtual machine.
3. Stop the helm-service
systemctl stop helm-service
4. Rename the binary to helm-service and copy the new binary to the following folder:
/opt/vmware/bin/
5. Add executable permissions to the binary
chmod 775 /opt/vmware/bin/helm-service
6. Start the helm-service
systemctl start helm-service
New environment variables:
- HELM_SERVICE_INSTALL_TIMEOUT: Timeout before failing installation jobs. (Default value is 5 minutes).
- HELM_SERVICE_UPGRADE_TIMEOUT: Timeout before failing upgrade jobs. (Default value is 5 minutes).
- HELM_SERVICE_UNINSTALL_TIMEOUT: Timeout before failing uninstall jobs. (Default value is 5 minutes).
- HELM_SERVICE_MAX_ROUTINES: The maximum number of parallel helm-service jobs to execute, additional jobs will be queued until a worker is available. (Default value is 10).
- HELM_SERVICE_WAITGROUP_ENABLED: Turns on/off the parallel worker limit, set to False to allow unlimited workers. (Default value is true).
To apply new environment variables:
Add the environment variables by modifying the systemd environment file located in the following folder of the TCA-CP
/etc/hybridity/helm-service.conf
Attachments
Feedback
Yes
No
Powered by
