Node Pool stuck in customizing state after MTU value change

Products

• VMware Telco Cloud Automation

Issue/Introduction

We see below error while Nodepolicy customization:

####-####-####-####-####    ####-nodepolicy-##       Provisioning    [
{"type": "NodepolicyApplied", "reason": "PolicyMcAccessError", "status": "False", "message": "Generate nodepolicies for generation 1 failed, reason:Get nodepolicy cr np1 from mc failed with newly created client is not active, error: newly created client is not active", "severity": "Error", "lastTransitionTime": "2025-01-11T00:00:00.####"} 

 

Environment

TCA 3.2

Cause

Also, when MTU configured with 3000 we see nodeconfig-operator daemonset with network access error:

capv@###-##-####-master-control-plane-#### [ ~ ]$ kubectl logs nodeconfig-daemon-### -n tca-system --previous
20##-##-##T##:##:##.###Z Created global stdout logWriter with level debug
20##-##-##T##:##:##.###Z [Info-nodeconfig-daemon] : k8sAPIRetryInterval is 1 seconds
20##-##-##T##:##:##.###Z [Info-nodeconfig-daemon] : Initializing daemon
20##-##-##T##:##:##.###Z [Info-profile_service] : Construct kube client
20##-##-##T##:##:##.###Z [Err-profile_service] : failed to construct k8s client: Get "https://10..#.#.#:443/api?timeout=32s": dial tcp 10..#.#.#:443: i/o timeout
20##-##-##T##:##:##.###Z [Err-nodeconfig-daemon] : failed to init daemon: Get "https://10..#.#.#:443/api?timeout=32s": dial tcp 10..#.#.#:443: i/o timeout 

Resolution

• You need to update MTU value in Overlay network to make ingress \ outgress able to handle the bandwidth with desired MTU value.
• To verify whether your network has MTU value, run ping command on workload cluster's worker node to test:

  • "ping -M do -s {MTU_value} {workload_cluster_VIP}"

• Sample output if your overlay network not configured with MTU value :

  • root@tcacp-####-####-####-####-####[ /home/admin ]# ping -M do -s 3000 10..#.#.#
    PING 10..#.#.# (10..#.#.#) 8800(8828) bytes of data.
    ping: local error: message too long, mtu=1500
    ping: local error: message too long, mtu=1500
    ping: local error: message too long, mtu=1500

• Sample output post your network update with desired MTU value:

  • capv@####-####-####-####-#### [ ~ ]$ ping -M do -s 2800 10..#.#.#
    PING 10..#.#.#(10..#.#.#) 2800(2828) bytes of data.
    2808 bytes from 10..#.#.#: icmp_seq=1 ttl=64 time=0.183 ms
    2808 bytes from 10..#.#.#: icmp_seq=7 ttl=64 time=0.132 ms