Failed to add VCF Operations component to VCF Fleet Inventory due to Error Code: 'LCMVROPSYSTEM25013'
search cancel

Failed to add VCF Operations component to VCF Fleet Inventory due to Error Code: 'LCMVROPSYSTEM25013'

book

Article ID: 441542

calendar_today

Updated On:

Products

VCF Operations

Issue/Introduction

  • Navigating to VCF Operations UI >> Fleet Management >> Lifecycle >> VCF Management returns below message:
    VCF Operations Fleet Management is Not Ready

  • Trying to re-connect the Fleet Management from VCF admin UI, we are getting below error:

    "Error connecting Fleet management node, try again"

  • Enabling the VCF Fleet UI and checking the "operations" component status shows "Deployment Failed"

  • Performing a retry on the Task from VCF Fleet UI, fails with below error code:

    Error Code : 'LCMVROPSYSTEM25013'
    com.vmware.vrealize.lcm.plugin.common.vrops.exceptions.InvalidCredentialException: Unauthorized access to Operations

  • The VCF Fleet "/var/log/vrlcm/vmware_vrlcm.log" contains below error:

    INFO vrlcm[1260] [pool-3-thread-19] [c.v.v.l.d.v.VropsConnector]  -- url to connect https://<VCF-Operations -IP>/casa/node/config
INFO vrlcm[1260] [pool-3-thread-19] [c.v.v.l.u.SSLUtil]  -- Disabled SNI for SSL socket.
INFO vrlcm[1260] [pool-3-thread-98] [c.v.v.l.a.g.s.EngineRequestServiceImpl]  -- Saving engine request: ########-####-####-####-############
INFO vrlcm[1260] [pool-3-thread-19] [c.v.v.l.u.CustomTrustManager]  -- Certificate chain trusted
INFO vrlcm[1260] [scheduling-1] [c.v.v.l.c.c.ContentDownloadController]  -- REPO_NAME :: /systemflowrepo  CONTENT_PATH :: /system/flow/vracreateserviceaccount.vmfx
INFO vrlcm[1260] [scheduling-1] [c.v.v.l.c.c.ContentDownloadController]  -- Decoded URL :: /systemflowrepo/system/flow/vracreateserviceaccount.vmfx
INFO vrlcm[1260] [pool-3-thread-19] [c.v.v.l.d.v.VropsConnector]  -- GET /casa/node/config: 401 401
ERROR vrlcm[1260] [pool-3-thread-19] [c.v.v.l.d.v.VropsEndpoint]  -- Unauthorized access to VMware Aria Operations
INFO vrlcm[1260] [scheduling-1] [c.v.v.l.c.c.ContentDownloadController]  -- ContentDTO{BaseDTO{vmid='vracreateserviceaccount', version=8.1.0.0} -> repoName='systemflowrepo', contentState='PUBLISHED', url='/systemflowrepo/system/flow/vracreateserviceaccount.vmfx'}
ERROR vrlcm[1260] [pool-3-thread-19] [c.v.v.l.p.c.v.t.BaseConfigureVropsTask]  -- admin password YXYXYXYX is invalid
INFO vrlcm[1260] [pool-3-thread-19] [c.v.v.l.p.a.s.Task]  -- Injecting task failure event. Error Code : 'LCMVROPSYSTEM25013', Retry : 'true', Causing Properties : 
'{ CAUSE :: vropsAdminPassword YXYXYXYX  }'
com.vmware.vrealize.lcm.plugin.common.vrops.exceptions.InvalidCredentialException: Unauthorized access to Operations
at com.vmware.vrealize.lcm.drivers.vrops.VropsEndpoint.getNodeState(VropsEndpoint.java:2687) ~[vmlcm-vropsplugin-driver-9.0.1.0-SNAPSHOT.jar!/:?]

     

  • The VCF Operations "/storage/log/vcops/log/casa/casa.log" reports below error:

    INFO  casa 18629 [ops@4413 threadId="2922145" threadName="ajp-nio-#.#.#.#-8011-exec-21" requestId="#######"] [com.vmware.vcops.casa.security.service.VcopsTokenAuthenticationService.generateCasaToken:115] - Provided token is expired
INFO  casa 18629 [ops@4413 threadId="2922145" threadName="ajp-nio-#.#.#.#-8011-exec-21" requestId="#######""] [com.vmware.vcops.casa.support.RequestIdIncomingInterceptor.afterCompletion:105] - Request POST /casa/authorize: Done
INFO  casa 18629 [ops@4413 threadId="2931680" threadName="pool-3-thread-32" requestId="#######"] [com.vmware.vcops.casa.security.SecurityService.syncAdminPassword:991] - No slices to call.
 INFO  casa 18629 [ops@4413 threadId="2931366" threadName="ajp-nio-#.#.#.#-8011-exec-25" requestId="#######"] [com.vmware.vcops.platform.utils.AdminUserUtils.checkCredentials:328] - Credentials invalid for user 'admin' - rejected
INFO  casa 18629 [ops@4413 threadId="2931366" threadName="ajp-nio-#.#.#.#-8011-exec-25" requestId="#######"] [com.vmware.vcops.casa.security.CasaAdminUserUtils.registerLoginResult:386] - Login failed for user 'admin': Attempt # 12, limit 5
INFO  casa 18629 [ops@4413 threadId="2931366" threadName="ajp-nio-127.0.0.1-8011-exec-25" requestId="#######"] [com.vmware.vcops.platform.utils.AdminUserUtils.writeCredentialsToFile:641] - Writing credentials for username 'admin' to /storage/vcops/user/conf/adminuser.properties
INFO  casa 18629 [ops@4413 threadId="2931366" threadName="ajp-nio-#.#.#.#-8011-exec-25" requestId="        "] [com.vmware.vcops.platform.utils.AdminUserUtils.isAccountLocked:295] - Lockout Time: 300000
INFO  casa 18629 [ops@4413 threadId="2931366" threadName="ajp-nio-#.#.#.#-8011-exec-25" requestId="        "] [com.vmware.vcops.platform.utils.AdminUserUtils.isAccountLocked:303] - Account 'admin' is locked.

INFO  casa 18629 [ops@4413 threadId="2931709" threadName="Thread-1539536" requestId="        "] [com.vmware.vcops.casa.fleetmanagement.lcm.LCMInternalService.doLcmCallWithResponse:108] - Submitting GET request to LCM node at https://<VCF-Fleet -FQDN>/lcm /request/api/v2/requests/########-####-####-####-############
INFO  casa 18629 [ops@4413 threadId="2931709" threadName="Thread-1539536" requestId="        "] [com.vmware.vcops.casa.fleetmanagement.lcm.LCMInternalService.doLcmCallWithResponse:162] - Request successful for GET request to https://<VCF-Fleet -FQDN>/request/api/v2/requests/########-####-####-####-############
INFO  casa 18629 [ops@4413 threadId="2931709" threadName="Thread-1539536" requestId="        "] [com.vmware.vcops.casa.fleetmanagement.FleetManagementNodeService.run:261] - ADD-FLEET-MANAGEMENT-NODE-WORKFLOW: LCM node state is: FAILED
ERROR casa 18629 [ops@4413 threadId="2931709" threadName="Thread-1539536" requestId="        "] [com.vmware.vcops.casa.fleetmanagement.FleetManagementNodeService.run:267] - Registration failed
INFO  casa 18629 [ops@4413 threadId="2931709" threadName="Thread-1539536" requestId="        "] [com.vmware.vcops.casa.fleetmanagement.FleetManagementNodeService.run:284] - ADD-FLEET-MANAGEMENT-NODE-WORKFLOW: Cleanup: Closing restTemplate

Environment

VCF Operations 9.x
VCF Fleet Management 9.x

Cause

The VCF Operations Fleet Management appliance disconnected and lost its registration with the primary VCF Operations instance.
This occurred because an internal token authentication session expired at the CaSA layer, which also blocked subsequent manual reconnection attempts.

Resolution

Follow the below steps to completed the registration of VCF Fleet with VCF Operations.

Additional Information

  • Run the following command on Fleet Management appliance as root user to enable the VCF Fleet Management UI to access Locker:
    touch /var/lib/vrlcm/UI_ENABLED
  • Delete the file in VCF Ops Fleet Management appliance to disable the VCF Ops Fleet management UI
    rm /var/lib/vrlcm/UI_ENABLED
Powered by Wolken Software