Follow Artifact Package distribution in the logs
search cancel

Follow Artifact Package distribution in the logs

book

Article ID: 400031

calendar_today

Updated On:

Products

CA Release Automation - Release Operations Center (Nolio) CA Release Automation - DataManagement Server (Nolio)

Issue/Introduction

In a deployment, a artifact package could be distributed from Retrieval Agent to the NES server and Agent

How to follow the distribution inside the logs ?
This could help in case of artifact package distribution failure or problem.

Environment

Release Automation - All Versions.

Resolution

STEP 1 - Download Artifact to retrieval Agent from NAC

On NAC in nolio_dm_all*.log, NAC sends a message to retrieval Agent to download the artifact package :

2025-06-05T11:13:32.316+02:00 [StageExecutorTask-4] INFO  (com.nolio.releasecenter.services.release.execution.engine.ArtifactPackageDistributionServiceImpl:97) - Downloading artifacts to retrieval agents for [Package2] for deployment [2025-06-05 DEPLOY1 ID.20]. 
2025-06-05T11:13:32.316+02:00 [StageExecutorTask-4] INFO  (com.nolio.releasecenter.services.release.execution.engine.ArtifactsEngineImpl:318) - collecting data to download [1] artifacts to retrieval agents for release id [20]
2025-06-05T11:13:32.322+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.server.dataservices.services.servers.AbstractServersServiceImpl:357) - Available agent[id=1, host=<Retrieval Agent>] found for agentGroup[id=5]
2025-06-05T11:13:32.330+02:00 [StageExecutorTask-4] INFO  (com.nolio.releasecenter.services.release.execution.engine.ArtifactsEngineImpl:482) - Artifact [ DEFINITION2.ZIP2'ver. 2.1'] Will be retrieved by the agent [<Retrieval Agent>]
2025-06-05T11:13:32.447+02:00 [StageExecutorTask-4] INFO  (com.nolio.platform.server.dataservices.services.artifact.ArtifactDistributionServiceImpl:42) - calling the Artifact executor with the mapping and the data [{es_<Retrieval Agent>={<Retrieval Agent>=[ArtifactDownloadData [taskType=DOWNLOAD, targetPath=files/ART20/20/3/arti.txt, moduleId=20, targetParentFolder=files/ART20, artifactId=3, artifactInstanceId=3, artifactFileName=arti.txt, artifactRepositoryPath=/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt, storeInRepository=true]]}}]
2025-06-05T11:13:32.479+02:00 [StageExecutorTask-4] INFO  (com.nolio.platform.server.dataservices.services.artifact.ArtifactDistributionServiceImpl:46) - Calling ArtifactServerExecutor

 

On Retrieval Agent in nolio_all.log, the download message is received and artifact package is downloaded :

2025-06-05T11:13:33.851+02:00 [New I/O server worker #1-2] DEBUG (com.nolio.platform.shared.communication.CommunicationNetworkServiceManager:212) - Message submitted for processing. Message: [ID:<uuid>_1@es_<NAC>, from:es_<NAC>, to:Artifact@<Retrieval Agent>- ArtifactDownloadData [taskType=DOWNLOAD, targetPath=files/ART20/20/3/arti.txt, moduleId=20, targetParentFolder=files/ART20, artifactId=3, artifactInstanceId=3, artifactFileName=arti.txt, artifactRepositoryPath=/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt, storeInRepository=true]], Service: com.nolio.platform.shared.communication.services.artifacts.ArtifactCommunicationService
2025-06-05T11:13:33.868+02:00 [ArtifactsHandlers-1] INFO  (com.nolio.platform.shared.communication.services.artifacts.ArtifactCommunicationService:127) - Starting artifact request
2025-06-05T11:13:33.870+02:00 [ArtifactsHandlers-1] INFO  (com.nolio.platform.shared.communication.services.artifacts.ArtifactCommunicationService:292) - Starting artifact service - downloadArtifact
2025-06-05T11:13:33.958+02:00 [ArtifactsHandlers-1] INFO  (com.nolio.platform.shared.storage.sync.filecache.SimpleFileCache:217) - SimpleFileCache won't add [files_registry\<uuid>], it already exists as [files_cache\<uuid>].
2025-06-05T11:13:33.958+02:00 [ArtifactsHandlers-1] INFO  (com.nolio.platform.shared.storage.registry.FilesRegistrySingleton:398) - FilesRegistry added file [files_registry\<uuid>] by id [RegistryFileId{bytes=<uuid>}]
2025-06-05T11:13:34.102+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:105) - Checking if artifact [/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt] exist in the repository [/nexus/content/repositories/nolio].
2025-06-05T11:13:34.128+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:432) - Creating HttpClient
2025-06-05T11:13:34.499+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:514) - Executing artifact upload request [GET http://<NAC_IP>:8080/nexus/content/repositories/nolio/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt HTTP/1.1].
2025-06-05T11:13:34.894+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:516) - Executed artifact upload request. Response: [HttpResponseProxy{HTTP/1.1 200  [Server: Unknown, X-Frame-Options: SAMEORIGIN, X-Content-Type-Options: nosniff, Accept-Ranges: bytes, ETag: "{SHA1{<uuid>}}", Last-Modified: Mon, 02 Jun 2025 15:37:55 GMT, Content-Type: text/plain, Content-Length: 6, Date: Thu, 05 Jun 2025 09:13:34 GMT, Keep-Alive: timeout=20, Connection: keep-alive] ResponseEntityProxy{[Content-Type: text/plain,Content-Length: 6,Chunked: false]}}].
2025-06-05T11:13:34.896+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:111) - Artifact path [/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt] was found in the configured repository [/nexus/content/repositories/nolio].
2025-06-05T11:13:34.898+02:00 [ArtifactsHandlers-1] DEBUG (com.nolio.platform.shared.communication.services.artifacts.ArtifactCommunicationService:297) - Artifact [/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt] was found in cache and was added to the files registry 

 

On NAC in nolio_dm_all*.log, NAC receives the download request from Retrieval agent and do the download

2025-06-05T11:13:35.144+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:105) - Checking if artifact [/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt] exist in the repository [/nexus/content/repositories/nolio].
2025-06-05T11:13:35.147+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:431) - Creating HttpClient
2025-06-05T11:13:35.150+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:513) - Executing artifact upload request [GET http://<NAC_IP>:8080/nexus/content/repositories/nolio/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt HTTP/1.1].
2025-06-05T11:13:35.185+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:515) - Executed artifact upload request. Response: [HttpResponseProxy{HTTP/1.1 200  [Server: Unknown, X-Frame-Options: SAMEORIGIN, X-Content-Type-Options: nosniff, Accept-Ranges: bytes, ETag: "{SHA1{<uuid>}}", Last-Modified: Mon, 02 Jun 2025 15:37:55 GMT, Content-Type: text/plain, Content-Length: 6, Date: Thu, 05 Jun 2025 09:13:35 GMT, Keep-Alive: timeout=20, Connection: keep-alive] ResponseEntityProxy{[Content-Type: text/plain,Content-Length: 6,Chunked: false]}}].
2025-06-05T11:13:35.185+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.shared.communication.services.artifacts.repositoryplugins.ArtifactRepositoryPlugin:111) - Artifact path [/4/DEFINITION2.ZIP2/2.1/DEFINITION2.ZIP2-2.1.txt] was found in the configured repository [/nexus/content/repositories/nolio].

 

 

STEP 2 - Distribution of Artifact package from Retrieval Agent to NES

On NAC in nolio_dm_all*.log, NAC sends a message to NES to do the distribution of artifact package :

2025-06-05T11:13:35.187+02:00 [StageExecutorTask-4] INFO  (com.nolio.releasecenter.services.release.execution.engine.ArtifactPackageDistributionServiceImpl:104) - Distributing the following artifacts [[Artifact [ DEFINITION2.ZIP2'ver. 2.1']]] to NESsfor deployment [2025-06-05 DEPLOY1 ID.20]. 
2025-06-05T11:13:36.440+02:00 [StageExecutorTask-4] INFO  (com.nolio.platform.server.dataservices.services.execmng.integration.HttpExecutionServerServicesFactory$ExecutionServerServices:60) - new ExecutionServerServices for server [<NES_NAME>]
2025-06-05T11:13:36.442+02:00 [StageExecutorTask-4] DEBUG (com.nolio.platform.server.dataservices.services.execution.control.executors.ExecutionServersExecutor:177) - submitting task [com.nolio.platform.server.dataservices.services.execution.control.tasks.ArtifactEsDistributionServerTask]. 

2025-06-05T11:13:36.522+02:00 [ExecutionServersWorker-3] DEBUG (org.apache.http.impl.conn.PoolingClientConnectionManager:185) - Connection request: [route: {}->http://<NES>:8080][total kept alive: 0; route allocated: 0 of 10; total allocated: 0 of 10]
2025-06-05T11:13:36.522+02:00 [ExecutionServersWorker-3] DEBUG (org.apache.http.impl.conn.PoolingClientConnectionManager:219) - Connection leased: [id: 226][route: {}->http://<NES>:8080][total kept alive: 0; route allocated: 1 of 10; total allocated: 1 of 10]
2025-06-05T11:13:37.477+02:00 [ExecutionServersWorker-3] DEBUG (org.apache.http.impl.conn.PoolingClientConnectionManager:268) - Connection [id: 226][route: {}->http://<NES>:8080] can be kept alive for 20000 MILLISECONDS
2025-06-05T11:13:37.477+02:00 [ExecutionServersWorker-3] DEBUG (org.apache.http.impl.conn.PoolingClientConnectionManager:275) - Connection released: [id: 226][route: {}->http://<NES>:8080][total kept alive: 1; route allocated: 1 of 10; total allocated: 1 of 10]

 

On NES in execution.log, NES receives the distribution message and download the file from Retrieval Agent

2025-06-05T11:13:37.650+02:00 [http-nio-8080-exec-1] INFO  (com.nolio.platform.server.execution.ExecutionEngineImpl:874) - Distributing artifacts to NES. Artifacts:[[NesDistributedArtifact{artifact_store/releaseId_20/ZIP2/DEFINITION2/2.1/arti.txt retriever:<Retrieval Agent>}]]

On NES in nimi.log :

2025-06-05T11:13:37.650+02:00 [http-nio-8080-exec-1] DEBUG (com.nolio.nimi.filetransfer.impl.FileTransferManagerImpl:409) - Making sure file with id `fid:<uuid>` gets written to `artifact_storeeleaseId_20\ZIP2\DEFINITION2\2.1\arti.txt`; hint source node is `nid:<Retrieval Agent>`.
2025-06-05T11:13:37.650+02:00 [http-nio-8080-exec-1] DEBUG (com.nolio.nimi.filetransfer.impl.FileTransferManagerImpl:467) - File id [fid:<uuid>] not found in fileInfoDB but found in cache.
2025-06-05T11:13:37.650+02:00 [http-nio-8080-exec-1] DEBUG (com.nolio.nimi.filetransfer.impl.FileTransferManagerImpl:470) - File id [fid:<uuid>] found cache, trying to register file use it from cache

 

 

STEP 3 - Distribution of Artifact package from NES to Agent

On Agent in nolio_all.log, Agent is downloading the artifact from NES

2025-06-05T11:13:39.762+02:00 [New I/O client worker #1-1] DEBUG (com.nolio.platform.shared.communication.CommunicationNetworkServiceManager:212) - Message submitted for processing. Message: [ID:<uuid>@es_<NES>, from:es_<NES>, to:ArtifactDistribution@<AgentName>- [NagDistributedArtifact{artifact_store/releaseId_20/ZIP2/DEFINITION2/2.1/arti.txt}]], Service: com.nolio.platform.shared.communication.services.artifacts.ArtifactDistributionService
2025-06-05T11:13:39.762+02:00 [Communication Msg Processor-4] INFO  (com.nolio.platform.shared.communication.services.artifacts.ArtifactDistributionService:40) - Distributing artifacts to this agent.
2025-06-05T11:13:39.762+02:00 [Communication Msg Processor-4] INFO  (com.nolio.platform.shared.communication.services.artifacts.ArtifactDistributionService:59) - Downloading artifact [artifact_store/releaseId_20/ZIP2/DEFINITION2/2.1/arti.txt] from NES [es_<NES>].
2025-06-05T11:13:39.858+02:00 [Communication Msg Processor-4] INFO  (com.nolio.platform.shared.storage.sync.filecache.SimpleFileCache:217) - SimpleFileCache won't add [files_registry\<UUID>], it already exists as [files_cache\<UUID>].
2025-06-05T11:13:39.858+02:00 [Communication Msg Processor-4] INFO  (com.nolio.platform.shared.storage.registry.FilesRegistrySingleton:461) - FilesRegistry added file [files_registry\<UUID>] by id [RegistryFileId{bytes=<UUID>}]


On NES in nimi.log, NES receives the download request from Agent

2025-06-05T11:13:40.096+02:00 [New I/O server worker #1-1] DEBUG (com.nolio.nimi.appmsg.durability.DurableCommunicationApi:265) - Received shipping: <AgentName>_174911402609103:payload=[ID:<uuid>_2@<AgentName>, from:<AgentName>, to:MESSAGE_RESPONSE_SERVICE@es_<NES>- [Response for message: <uuid>_2@es_<NES>]]
2025-06-05T11:13:40.112+02:00 [JobExecutorThread-2] DEBUG (com.nolio.nimi.appmsg.durability.DurableCommunicationApi:197) - Got new message: es_<NES>_174911402625404:payload=[ID:<uuid>_3@es_<NES>, from:es_<NES>, to:ArtifactDistribution@<AgentName>- [NagDistributedArtifact{artifact_store/releaseId_20/ZIP2/DEFINITION2/2.1/arti.txt}]]

 

Powered by Wolken Software