Symptoms:
- You see the message "failed to apply Harbor config" when applying changes to the Harbor configuration.
- This issue occurs if this is the task is performed from the Enterprise PKS Management Console (EPMC) or Opsmanager.
- Harbor is using NFS-backed storage for harbor-registry.
- Once failed, the Harbor deployment VM is in a stopped state.
- monit summary shows no process running in the Harbor vm
- You see messages similar to the following from the bosh task command (run against the bosh task ID related to the Harbor configuration change):
Task 1854 | 22:04:51 | Updating instance harbor-app: harbor-app/2becda9c-b215-48f4-b932-7734f4c6a4b2 (0) (canary) (00:01:37)
L Error: Action Failed get_task: Task e0ce7d47-7741-4812-6499-319e7231ffca result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 1854 | 22:06:28 | Error: Action Failed get_task: Task e0ce7d47-7741-4812-6499-319e7231ffca result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 1854 Started Thu Feb 27 22:04:48 UTC 2020
Task 1854 Finished Thu Feb 27 22:06:28 UTC 2020
Task 1854 Duration 00:01:40
Task 1854 error
Capturing task '1854' output:
Expected task '1854' to succeed but state is 'error'
Exit code 1
- You see messages similar to the following in the /var/vcap/sys/log/harbor/pre-start.stderr.log file:
Removing network harbor-app_harbor-clair
Removing network harbor-app_harbor-notary
Removing network harbor-app_harbor-chartmuseum
Removing network harbor-app_notary-sig
Warning: Stopping rpcbind.service, but it can still be activated by:
rpcbind.socket
mount.nfs: /harbor-registry is busy or already mounted
Warning: Stopping rpcbind.service, but it can still be activated by:
rpcbind.socket
mount.nfs: /harbor-registry is busy or already mounted
(END)
- You see an error similar to the following if you try to manually start the Harbor VM via the bosh command:
Task 1871 | 22:24:18 | Updating instance harbor-app: harbor-app/2becda9c-b215-48f4-b932-7734f4c6a4b2 (0) (canary) (00:00:55)
L Error: Action Failed get_task: Task 041411e1-88ed-4890-6772-fe24064dc7f3 result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 1871 | 22:25:13 | Error: Action Failed get_task: Task 041411e1-88ed-4890-6772-fe24064dc7f3 result: 1 of 4 pre-start scripts failed. Failed Jobs: harbor. Successful Jobs: enable-bosh-dns, bosh-dns, wavefront.
Task 1871 Started Thu Feb 27 22:24:17 UTC 2020
Task 1871 Finished Thu Feb 27 22:25:13 UTC 2020
Task 1871 Duration 00:00:56
Task 1871 error
Changing state:
Expected task '1871' to succeed but state is 'error'
Exit code 1