Content Library & Subscribed content libraries failing to deploy after sync - Checking for missing/corrupted data issues

Products

• VMware vCenter Server

Issue/Introduction

• Source content library is failing deployment of some or all of its Templates, Images or OVA's/OVF's
• Remote content library starts to fail deployments after syncing with source library.

Environment

• Deployments with multiple subscribed content libraries.
• VMware vSphere

Cause

• Confirm a new content library is able to perform deployments.
• Try to download contents from the affected content library any item that failed to deploy, these should not download correctly or download with a 0 size. (Confirm download size is correct as a simple corrupt data verification, additional check using CRC/MD5 check against original source file if practical).
• If these do not download correctly please check logs for the content library directly including "cls.log" and look for the contents paths. Example as below: 
  • A general system error occurred: Error retrieving files for library item VM######-####-####-####-OBJECT#####..
  • Reason:VM-NAME-#-###-####-##-disk-#.vmdk: target 'ds:///vmfs/volumes/data####-store###-####-############//contentlib-#######-####-####-####-############/########-#####-####-####-#########/VM-NAME-#-###-####-##-disk-#_#######-####-####-####-############.vmdk?serverId=########-####-####-####-############'error: NFC Error 1/16: No file exists for given path -- File open failed: The system cannot find the file specified.
• Check these paths on the content library datastore, they will either be missing or have missing files.

Resolution

• If there is a remote subscribed library that was not synced after the issue use it to download all contents and to then re-create the content library. (This will only work if there are no automated syncs and the library was fully synced before the issue)
• If downloading the contents is not possible a new content library will need to be created from scratch from new sources.

Additional Information

• This will happen due to removal of contents configured on the content library directly from the datastores instead of performing the removal from the Content Library itself. Please do not remove content library items manually.
• For more information refer to the docs: Using Content Libraries