"Reason: No reason given" user permission added to VC is removed automatically upon restarting vpxd service after upgrading to VC 6.7.

Products

• VMware vCenter Server

Issue/Introduction

• User permission added are removed automatically from vCenter server upon service restart with "Reason: No reason given" after upgrading to vCenter Server 6.7.

• In /var/log/vmware/vpxd/vpxd.log or %ALLUSERSPROFILE%\VMWare\vCenterServer\logs\vmware-vpx\vpxd.log, you see entries similar to:

YYYY-MM-DDTHH:MM:SS.860-05:00 error vpxd[04162] [Originator@6876 sub=AuthorizeManager] Removing invalid permission 333: unknown error
YYYY-MM-DDTHH:MM:SS.860-05:00 warning vpxd[04162] [Originator@6876 sub=Default] Removing permission for entity "group-v2760", group "VSPHERE.LOCAL\user_details", role 721420390.  Reason: No reason given

 
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vCenter Server Appliance 6.7.x

Cause

This issue occurs because vCenter Server is expecting the administrator to be loaded as first permission before any other permissions are added while checking the postgres database.

Resolution

This issue has been resolved in vCenter Server 6.7 U1b/EP6, you can download the vCenter Server 6.7U1b VMware Downloads.

Workaround:

Caution:  Please take snapshots of the vCenter server before we proceed further.

To workaround this issue, re-order the permission list using one of the method below:

Manual Method:

1. Find the permission entry list of Root object from vpx_access table and make a note of all entries with entity_id IS 1 and role_id IS NOT -1 above the first role_id -1 in the list

• On vCenter Server Appliance: 

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "select * from VPX_ACCESS where entity_id = '1';"

• On vCenter Server installed on Windows (Connecting to the embedded vPostgres Database in a Windows installed vCenter Server 6.x): 

select * from VPX_ACCESS where entity_id = '1' 

• On MS SQL: 

select * from VPX_ACCESS where entity_id = '1'

Example:
# /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "select * from VPX_ACCESS where entity_id = '1';"
  id  |              principal              |  role_id  | entity_id | flag | surr_key
------+-------------------------------------+-----------+-----------+------+----------
   xx| VSPHERE.LOCAL\user_details          | 721420390 |         1 |    3 |        2
   yy| VSPHERE.LOCAL\Administrator         |        -1 |         1 |    1 |      103
(2 rows)

Expected : Root object (entity_id 1) should have a user with role_id -1 as first in the list.

2.  Remove and re-add all entries with entity_id IS 1 and role_id IS NOT -1 to re-order the list:
   1. From vSphere Web Client Host and Clusters Navigator, select the vCenter Server object (Root of the inventory) 
   2. Go to permissions tab
   3. Note the role of User/Groups identified in step 1 
   4. Remove the permission
   5. Re-add the permission back
3. Validate the permission order from Database:

• On vCenter Server Appliance: 

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "select * from VPX_ACCESS where entity_id = '1';"

• On vCenter Server installed on Windows (Connecting to the embedded vPostgres Database in a Windows installed vCenter Server 6.x): 

select * from VPX_ACCESS where entity_id = '1' 

• On MS SQL:

  select * from VPX_ACCESS where entity_id = '1'

Example:
# /opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB -c "select * from VPX_ACCESS where entity_id = '1';"
  id  |              principal              |  role_id  | entity_id | flag | surr_key
------+-------------------------------------+-----------+-----------+------+----------
 yy| VSPHERE.LOCAL\Administrator         |        -1 |         1 |    1 |      103
 zz| VSPHERE.LOCAL\user_details          | 721420390 |         1 |    2 |      104
(2 rows)

4. After this re-order, new permissions added will be available even after vCenter Server Service restart.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Automated Method for vCenter Server Appliance:

1. Copy the attached script restore_permissions.sh to vCenter Server Appliance
2. Add execution privilege for the script:

chmod +x restore_permissions.sh

3. Execute the script

./restore_permissions.sh <current administrator> <desired full administator>

<current administrator> - Existing user who has Administrator privilege on vCenter Server to make privilege changes
<desired full administator> - The user who has existing privilege entry on vCenter Server and should be the desired Root Object Administrator to re-order and bring on top