Migrating TKGI cluster to the NSX Policy API fails with error:- no such file or directory
search cancel

Migrating TKGI cluster to the NSX Policy API fails with error:- no such file or directory

book

Article ID: 380486

calendar_today

Updated On:

Products

VMware Tanzu Kubernetes Grid Integrated Edition 1.x VMware Tanzu Kubernetes Grid Integrated (TKGi)

Issue/Introduction

While migrating cluster to the NSX Policy API using tkgi promote-cluster-to-policy <cluster-name> it might fail. 

#tkgi cluster <cluster-name>

PKS Version: 1.20.0-build.10
Name: test-cluster
K8s Version: 1.29.4
Plan Name: small
UUID: 544412ce-0e02-44d7-b1b2-789793ba184d
Last Action: MIGRATE_NSX
Last Action State: failed
Last Action Description: Instance running errands failed: There was a problem completing your request. Please contact your operations team providing the following information: service: p.pks, service-instance-guid: 544412ce-0e02-44d7-b1b2-789793ba184d, broker-request-id: 892c9ab1-122f-445d-82fe-9ee2d8205525, task-id: 286947, operation: errands-update, error-message: Action Failed get_task: Task 4c11b45e-ef7c-4757-7472-d564d083a3c1 result: Copying filtered files to temp directory: Copying Files to Temp Dir: Getting file info for '/var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1': stat /var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1: no such file or directory
Kubernetes Master Host: test-cluster
Kubernetes Master Port: 8443
Worker Nodes: 10
NSX Policy: false
Tags:

Bosh task fails at fetching logs step

# bosh task 286947
Using environment '192.168.1.21' as client 'ops_manager'

Task 286947

Task 286947 | 21:55:32 | Preparing deployment: Preparing deployment
Task 286947 | 21:55:32 | Deprecation: Global 'properties' are deprecated. Please define 'properties' at the job level.
Task 286947 | 21:55:35 | Preparing deployment: Preparing deployment (00:00:03)
Task 286947 | 21:55:35 | Running errand: master/1451e2c5-872e-4740-b52b-69b928b4ab66 (0) (00:50:44)
Task 286947 | 22:46:19 | Fetching logs for master/1451e2c5-872e-4740-b52b-69b928b4ab66 (0): Finding and packing log files (00:00:09)
                       L Error: Action Failed get_task: Task 4c11b45e-ef7c-4757-7472-d564d083a3c1 result: Copying filtered files to temp directory: Copying Files to Temp Dir: Getting file info for '/var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1': stat /var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1: no such file or directory
Task 286947 | 22:46:28 | Error: Action Failed get_task: Task 4c11b45e-ef7c-4757-7472-d564d083a3c1 result: Copying filtered files to temp directory: Copying Files to Temp Dir: Getting file info for '/var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1': stat /var/vcap/sys/log/migrate-mp2p/migrate-mp2p.stdout.log.1: no such file or directory

Task 286947 Started  Mon Sep 16 21:55:32 UTC 2024
Task 286947 Finished Mon Sep 16 22:46:28 UTC 2024

Although on the master node file exists.

master/1451e2c5-872e-4740-b52b-69b928b4ab66:/var/vcap/sys/log/migrate-mp2p# ls
all_resources_42022.log  migrate-mp2p.stdout.log  migrate-mp2p.stdout.log.1.gz  mp_to_policy_import.log  run.stderr.log  run.stdout.log

master/1451e2c5-872e-4740-b52b-69b928b4ab66:/var/vcap/sys/log/migrate-mp2p# stat migrate-mp2p.stdout.log.1.gz
  File: migrate-mp2p.stdout.log.1.gz
  Size: 12062740      Blocks: 23568      IO Block: 4096   regular file
Device: 812h/2066d    Inode: 261754      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-09-18 07:18:49.605227770 +0000
Modify: 2024-09-16 22:46:03.762527485 +0000
Change: 2024-09-16 22:46:20.931955564 +0000
 Birth: 2024-09-16 22:46:18.679768236 +0000
master/1451e2c5-872e-4740-b52b-69b928b4ab66:/var/vcap/sys/log/migrate-mp2p# stat migrate-mp2p.stdout.log
  File: migrate-mp2p.stdout.log
  Size: 0             Blocks: 0          IO Block: 4096   regular empty file
Device: 812h/2066d    Inode: 261751      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-09-18 07:18:49.601227434 +0000
Modify: 2024-09-16 22:46:18.659766572 +0000
Change: 2024-09-16 22:46:18.659766572 +0000
 Birth: 2024-09-16 22:14:54.772147712 +0000

Environment

TKGi 1.20

Cause

Failure can occur due to race condition with logrotate when the files are being fetching during the log rotation process. 

Resolution

Executing the command tkgi promote-cluster-to-policy <cluster-name> again will resolve the issue.

Powered by Wolken Software