Harbor UI fails with "Core service is not available" due to Version Mismatch

Products

• VMware Telco Cloud Automation

Issue/Introduction

• The Harbor Registry UI is inaccessible.
• The browser displays the error message: Core service is not available.
• The harbor-core container is in a Starting or Unhealthy state.
• The redis container is in a continuous Restarting loop.
• The Redis logs (docker logs redis) display the following error:
  Can't handle RDB format version 10
Fatal error loading the DB: Invalid argument. Exiting.
• The running Docker images (checked via docker ps or docker images) show an older version (e.g., v2.3.3), while a newer installation (e.g., v2.6.x) exists on the file system.

Environment

• VMware Telco Cloud Automation (TCA) 3.2
• VM-based Harbor Registry (Docker Compose deployment)
• Harbor version 2.x

Cause

• This issue is caused by a configuration drift in the system startup files following a Harbor upgrade.
• The systemd service file (typically /etc/systemd/system/harbor.service) is still configured to point to the old Harbor installation directory (e.g., /opt/harbor). However, the data on the disk (specifically the Redis database) has already been upgraded to a newer format compatible only with the new Harbor version.
• When the VM reboots, the system launches the old Harbor binaries. These old binaries crash because they are not backward compatible with the newer data formats found on the disk (Redis RDB Format 10).

Resolution

To resolve this issue, you must update the systemd service configuration to point to the correct, newer Harbor installation directory and verify that the service persists after a reboot.

1. Stop the Legacy Containers Stop and remove the currently running incompatible containers to release system locks and ports.

# Stop all running containers
docker stop $(docker ps -q)

# Prune stopped containers to ensure a clean slate
docker container prune -f

2. Verify the Correct Installation Path Locate the directory containing the docker-compose.yml file for the new version (e.g., v2.6.3).

# Example command to find the correct configuration
find / -name "harbor.yml"

Take note of this path (e.g., /home/vmware/harbor-v2.6.3/harbor).

3. Update the Systemd Service Edit the Harbor service file to reflect the correct path.

vi /etc/systemd/system/harbor.service

Update the WorkingDirectory, ExecStart, and ExecStop parameters:

[Service]
# Update this to your NEW installation path
WorkingDirectory=/home/vmware/harbor-v2.6.3/harbor

# Ensure the Exec commands point to the correct compose file
ExecStart=/usr/bin/docker compose -f /home/vmware/harbor-v2.6.3/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker compose -f /home/vmware/harbor-v2.6.3/harbor/docker-compose.yml down

(Note: Adjust /usr/bin/docker compose to /usr/local/bin/docker-compose if you are using the older standalone binary).

4. Apply Changes Reload the systemd daemon to recognize the changes.

systemctl daemon-reload

5. Reboot and Verify Persistence Reboot the Virtual Machine to ensure the new configuration applies automatically and the fix is persistent.

reboot

6. Final Validation Once the VM is back online, wait approximately 60 seconds and check the container status to confirm the new version started automatically.

# Verify the service is active
systemctl status harbor

# Verify the container versions match the upgrade (e.g., v2.6.3)
docker ps

Ensure that the redis and harbor-core containers are listed as Up (healthy).

Additional Information

• Redis Compatibility: Redis database files are version-specific. An older Redis binary cannot read a dump file created by a newer Redis binary.
• Backup: Always backup your harbor.service file before editing: cp /etc/systemd/system/harbor.service /etc/systemd/system/harbor.service.bak