Users may lose content from content libraries backed by SMB share when upgrading from vSphere 6.5 to vSphere 6.7

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
After upgrading vSphere 6.5 to vSphere 6.7, users might not be able to access the content of some libraries that are backed by SMB share.

Cause

Content Library does not detect SMB mount error when creating libraries backed by SMB shares in vSphere 6.5, therefore such libraries can be created without any error and uploading content to such libraries will consume storage on vCenter server. When upgrading from vSphere 6.5 to vSphere 6.7, the content of such libraries will not be migrated, and they won't be accessible from content library after the upgrade.

Resolution

To resolve this issue, perform the workaround steps provided in KB 58932 before upgrading to 6.7.

In case you still have libraries backed by SMB shares with content on vCenter Server during the upgrade, perform the following steps to recover the content of such libraries after upgrade:

1. Power down the upgraded target vCenter Server (6.7.x) and power on the source vCenter Server (6.5.x)

2. Find out the libraries backed by SMB shares by running the following SQL query against VCDB of the source vCenter (6.5.x):

select lib.id, lib.name, storage.storageuri from cl_library lib, cl_library_storage lib_storage, cl_storage storage where lib.id = lib_storage.library_id and lib_storage.storage_id = storage.id and storage.storageuri like 'file:///etc/vmware-content-library/winmounts/library-win-mount%';

Sample Result:
                  id                  |  name  |                                                                                       storageuri                                                                    
--------------------------------------+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------
########-####-####-####-##########88 | smblib | file:///etc/vmware-content-library/winmounts/library-win-mount-########-####-####-####-##########88-ClsPathSeparator-smb://cls-qe-testdata.eng.vmware.com/storage/cl_qe/smbBackupMount/

3. For each library from the list returned by the SQL query, run the following command on the source vCenter server to find out if the smb mount is actually on local file system, and add the affected libraries to a SMB library list:

(a) Get the mount path on file system from the storage URL (the last column) in the SQL query result:

sample command:
echo "file:///etc/vmware-content-library/winmounts/library-win-mount-########-####-####-####-##########88-ClsPathSeparator-smb://cls-qe-testdata.eng.vmware.com/storage/cl_qe/smbBackupMount/" | sed 's/file:\/\/\(.*\)-ClsPathSeparator-smb:.*/\1/'
/etc/vmware-content-library/winmounts/library-win-mount-########-####-####-####-##########88

(b) Check if the mount path is actually mounted on the server:

Sample command:
mount | grep "/etc/vmware-content-library/winmounts/library-win-mount-########-####-####-####-##########88"

If this command returns no entry (null), it means the library is consuming storage space from the vCenter server, and then add it to the SMB library list.

4. For each SMB library in the SMB library list, download all item files locally and organize them per library accordingly.

5. Power off the source vCenter (6.5.x) and power on the target vCenter (6.7.x)

6. For each SMB library in the SMB library list, perform the following actions to restore content on the target vCenter (6.7.x):
 

   a. Create a new library on the target vCenter (6.7.x) with storage on datastore, or NFS share if applicable.
   b. Upload all item files to the library that are downloaded in step 4.
   c. Now all the content are restored to the upgraded vCenter (6.7.x), and the affected SMB library can be deleted.