#> Updating deployment:
Expected task '58476' to succeed but state is 'error'
Exit code 1
Task 58476 | 09:40:40 | L executing pre-start: harbor-app/ed1d446d-7b0e-4857-a670-b08a2cda58bf (0) (canary) (00:00:19)
L Error: Action Failed get_task: Task 5c822ca2-bacf-4bc1-5f82-db8f711b6bc3 result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 58476 | 09:40:46 | Error: Action Failed get_task: Task 5c822ca2-bacf-4bc1-5f82-db8f711b6bc3 result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 58476 Started Tue Aug 29 09:40:21 UTC 2023
Task 58476 Finished Tue Aug 29 09:40:46 UTC 2023
Task 58476 Duration 00:00:25
Task 58476 error
===== 2023-08-29 09:40:46 UTC Finished "/usr/local/bin/bosh --no-color --non-interactive --tty --environment=10.68.96.5 --deployment=harbor-container-registry-09d28774917ffa47fe22 deploy --no-redact /var/tempest/workspaces/default/deployments/harbor-container-registry-09d28774917ffa47fe22.yml"; Duration: 35s; Exit Status: 1
Exited with 1.
Exited with 1.
{code:java}
harbor-app/ed1d446d-7b0e-4857-a670-b08a2cda58bf:/var/vcap/jobs/harbor/bin# openssl s_client -showcerts -servername api.np-pksapi.globetel.com -connect api.np-pksapi.globetel.com:8443
gethostbyname failure
connect:errno=0
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/var/vcap/packages/python/python2.7/lib/python2.7/json/__init__.py", line 291, in load
**kw)
File "/var/vcap/packages/python/python2.7/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/var/vcap/packages/python/python2.7/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/var/vcap/packages/python/python2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (60) server certificate verification failed. CAfile: /var/vcap/jobs/harbor/config/uaa_ca.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html
The Harbor instance is configured with UAA authentication, it requires connecting to the UAA server first in the pre-start script, but it failed to establish a TLS connection with the UAA server because the "uaa_ca.crt" is wrong.
Because the UAA certificate is rotated in server side, the CA certificate in Harbor tile should be reconfigured with the new UAA CA cert.
1. Copy the UAA cert from /var/vcap/jobs/uaa/config/
bosh -d pivotal-container-service-xxxx ssh pivotal-container-service/xxxx
sudo su # switch root user
cat /var/vcap/jobs/uaa/config/uaa.crt
2. Remove the private key section from uaa.crt (if exists) and paste the uaa.crt as a CA under Harbor tile "certificate" section.
3. Then click "Apply Changes" on Harbor tile .
4. After successful "Apply changes" , verify that UAA user can now login to Harbor console.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Failed 5m12s (x307 over 75m) kubelet, xxxxx.compute.internal Error: ImagePullBackOff
Normal BackOff <invalid> (x419 over 75m) kubelet, xxxxx.compute.internal Back-off pulling image "harbor-xxxxx"