SCS Config Server Label Not Found
Article ID: 297188
Updated On:
Products
Support Only for Spring
Issue/Introduction
This Knowledge Base (KB) article details a reason for why a Spring Cloud Services (SCS) config-server Service Instance (SI) may not be able to find its configured git label.
A config-server is a central component that manages configuration settings for applications, promoting consistency and easy updates across the system. More details about SCS config-servers can be found in the documentation.
When setting up a config-server with Git, you specify various configuration details. This includes an external git repository URI, which allows the SCS mirror service to clone the target repo to the local mirror. You also define a Git label, which represents the target branch from the external repo to configure the config-server with. If a config-server is created with a label not found in the local mirror, an error is thrown (e.g., using the label "develop").
If the label configured for the config-server exists in the upstream git repo, but the mirror service does not have it in the local mirror, then the config-server may just need its mirrors refreshed to fetch the new labels. This is an expected behavior due to the internal mechanics that the mirror service uses to manage its local mirrors.
A config-server is a central component that manages configuration settings for applications, promoting consistency and easy updates across the system. More details about SCS config-servers can be found in the documentation.
When setting up a config-server with Git, you specify various configuration details. This includes an external git repository URI, which allows the SCS mirror service to clone the target repo to the local mirror. You also define a Git label, which represents the target branch from the external repo to configure the config-server with. If a config-server is created with a label not found in the local mirror, an error is thrown (e.g., using the label "develop").
2023-11-20T11:54:33.74-0500 [APP/PROC/WEB/0] OUT [http-nio-8080-exec-8] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet].log - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.cloud.config.server.environment.NoSuchLabelException: No such label: develop] with root cause 2023-11-20T11:54:33.74-0500 [APP/PROC/WEB/0] OUT org.eclipse.jgit.api.errors.RefNotFoundException: Ref develop cannot be resolved
If the label configured for the config-server exists in the upstream git repo, but the mirror service does not have it in the local mirror, then the config-server may just need its mirrors refreshed to fetch the new labels. This is an expected behavior due to the internal mechanics that the mirror service uses to manage its local mirrors.
Environment
Product Version: 3.1
Resolution
In order to understand why this would happen, a high level understanding of mirror service is required. The mirror service is hosted on the SCS VM and local mirrors are saved to the persistent disk at the following location:
/var/vcap/store/mirror/<mirror-ID>/<repository>/
When a config-server configured with git is created, mirror service considers if it already has a local mirror for the provided external git repository URI.
If mirror service discovers it does not have a local mirror for the provided repo, then it will create a new one as indicated from its logs:
If mirror service discovers it does have a local mirror for the provided repo, then it will reuse the existing mirror as indicated from its logs:
Based on the above information, the following sequence of events can result in the scenario where a label configured for a config-server exists in the upstream git repo, but the mirror service does not have it in the local mirror:
1 - Create config-server-A pointing to repo-A branch branch-A and no periodic refresh.
2 - Create branch-B in repo-A in the external git repo.
3 - Create config-server-B pointing to repo-A branch branch-B and no periodic refresh.
When config-server-B is created, the mirror service will realize it already has a mirror for repo-A. If the local mirror on the SCS VM for this repo has not been refreshed since config-server-A's creation, then the mirror will not have branch-B and config-server-B will face errors until the mirror is refreshed. Refreshing the mirror will result in a "git fetch" command being ran, which will then pull in the new branch.
After refreshing the config-server mirror we can the new branch is now discovered:
In conclusion, mirror service will maintain one local mirror per remote repo. If many config-servers are created pointing to the same remote repo, then they will share the same internal mirror within mirror service. To make sure the local mirror is in sync with the remote repo, refreshing the mirror is encouraged. There are 5 methods for refreshing a mirror.
1 - cf config-server-sync-mirrors my-config-server via the cf CLI plugin
2 - Manual sync via dashboard
3 - cf update-service my-config-server -c ‘{“update-git-repos”: true }’
4 - Adding the parameter periodic=true to the config-server configuration. This feature is available from SCS v3.1.2+
5 - Hitting the config-server's actuator endpoint:
If mirror service discovers it does not have a local mirror for the provided repo, then it will create a new one as indicated from its logs:
2023-11-20T16:38:23.439Z INFO 5 --- [ctor-http-nio-4] s.c.m.b.DelegatingServiceInstanceService : Creating service instance with id 2938e7d1-0257-4b11-82d0-5fe265bf3517 2023-11-20T16:38:23.442Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.m.broker.MirrorManager : Creating new mirror for [email protected]:pivotal-Josh-Gainey/my-scs-test-repo 2023-11-20T16:38:23.442Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.m.mirrors.ShellSshMirrorServer : Creating mirror 55ff81b3084dc342f02cd7a74a4c0155/my-scs-test-repo 2023-11-20T16:38:23.443Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : Running shell command: git -c "core.sshCommand=ssh -i /var/vcap/store/mirror/55ff81b3084dc342f02cd7a74a4c0155/private-key -o StrictHostKeyChecking=no" clone --mirror [email protected]:pivotal-Josh-Gainey/my-scs-test-repo /var/vcap/store/mirror/55ff81b3084dc342f02cd7a74a4c0155/my-scs-test-repo 2023-11-20T16:38:24.980Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : Cloning into bare repository '/var/vcap/store/mirror/55ff81b3084dc342f02cd7a74a4c0155/my-scs-test-repo'...
If mirror service discovers it does have a local mirror for the provided repo, then it will reuse the existing mirror as indicated from its logs:
2023-11-20T16:42:05.462Z INFO 5 --- [ctor-http-nio-2] s.c.m.b.DelegatingServiceInstanceService : Creating service instance with id aa901d63-75b6-4f8b-a4b3-e9543ccfbbe6 2023-11-20T16:42:05.467Z INFO 5 --- [ctor-http-nio-2] i.p.s.c.m.broker.MirrorManager : Mirror for [email protected]:pivotal-Josh-Gainey/my-scs-test-repo already exists
Based on the above information, the following sequence of events can result in the scenario where a label configured for a config-server exists in the upstream git repo, but the mirror service does not have it in the local mirror:
1 - Create config-server-A pointing to repo-A branch branch-A and no periodic refresh.
2 - Create branch-B in repo-A in the external git repo.
3 - Create config-server-B pointing to repo-A branch branch-B and no periodic refresh.
When config-server-B is created, the mirror service will realize it already has a mirror for repo-A. If the local mirror on the SCS VM for this repo has not been refreshed since config-server-A's creation, then the mirror will not have branch-B and config-server-B will face errors until the mirror is refreshed. Refreshing the mirror will result in a "git fetch" command being ran, which will then pull in the new branch.
After refreshing the config-server mirror we can the new branch is now discovered:
2023-11-20T20:44:54.545Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.m.refresh.RefreshEndpoint : Refreshing Git mirrors for service instance with id aa901d63-75b6-4f8b-a4b3-e9543ccfbbe6 2023-11-20T20:44:54.794Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.m.refresh.RefreshExecutor : Updating mirror repo with source uri [email protected]:pivotal-Josh-Gainey/my-scs-test-repo 2023-11-20T20:44:54.797Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : Running shell command: git -c "core.sshCommand=ssh -i /var/vcap/store/mirror/55ff81b3084dc342f02cd7a74a4c0155/private-key -o StrictHostKeyChecking=no" fetch origin 2023-11-20T20:44:56.340Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : From github.com:pivotal-Josh-Gainey/my-scs-test-repo 2023-11-20T20:44:56.340Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : * [new branch] develop -> develop 2023-11-20T20:44:56.340Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : Running shell command: git rev-parse HEAD 2023-11-20T20:44:56.344Z INFO 5 --- [ctor-http-nio-4] i.p.s.c.mirrorservice.mirrors.ShellUtil : e76dde7188a2d6f927f9ea8d91efdd8cc7999461
In conclusion, mirror service will maintain one local mirror per remote repo. If many config-servers are created pointing to the same remote repo, then they will share the same internal mirror within mirror service. To make sure the local mirror is in sync with the remote repo, refreshing the mirror is encouraged. There are 5 methods for refreshing a mirror.
1 - cf config-server-sync-mirrors my-config-server via the cf CLI plugin
2 - Manual sync via dashboard
3 - cf update-service my-config-server -c ‘{“update-git-repos”: true }’
4 - Adding the parameter periodic=true to the config-server configuration. This feature is available from SCS v3.1.2+
5 - Hitting the config-server's actuator endpoint:
curl -X POST -H "Authorization: $(cf oauth-token)" -H "Content-Type: application/json" https://config-server-a5782192-8036-4f57-8312-4756a2604240.apps.example.com/actuator/refreshmirrors
Feedback
Yes
No
Powered by
