Opensearch service experiences start delay or file permission errors due to incorrect log directory ownership

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

When starting the opensearch service, you may observe a long delay before the service starts, or the service may start but immediately output file permission errors in the console. This issue occurs when executing the start command, as shown in the example below:

root@<hostname> [ ~ ]# /etc/init.d/opensearch start
horizon-workspace service is running
Waiting for IDM: Ok.
Number of nodes in cluster is : 3
Configuring /opt/vmware/opensearch/config/opensearch.yml file
Starting opensearch: vm.max_map_count = 262144
done.
    opensearch logs: /opt/vmware/opensearch/logs
    opensearch data: /db/opensearch
root@<hostname> [ ~ ]# <date> <time> main ERROR Could not define attribute view on path "/opt/vmware/opensearch/logs/horizon_deprecation.log" got access denied ("java.lang.RuntimePermission" "lookupUserInformation") java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "lookupUserInformation")
...
<date> <time> main ERROR Could not define attribute view on path "/opt/vmware/opensearch/logs/horizon_deprecation.json" got access denied ("java.lang.RuntimePermission" "lookupUserInformation") java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "lookupUserInformation")
...

Environment

VMware Identity Manager 3.3.7

Cause

The issue is caused by the group ownership of some files within the /opt/vmware/opensearch/logs directory being incorrectly set to "users" instead of the required group, which is "www".

The incorrect ownership appears as follows (example file list):

-rw------- 1 opensearch users 763 Month Day Time horizon_deprecation.json
-rw------- 1 opensearch users 535 Month Day Time horizon_deprecation.log

Resolution

The resolution is to correct the file ownership of the opensearch log directory on all affected nodes.

Run the following steps on all nodes where the Opensearch service is running.

1. Stop the opensearch service on the node:

   /etc/init.d/opensearch stop

2. Change the ownership (user and group) of the opensearch log directory and its contents recursively to opensearch:www:

   chown -R opensearch:www /opt/vmware/opensearch/logs

3. Restart the opensearch service:

   /etc/init.d/opensearch start

Once the ownership is corrected and the service is restarted, the permission errors should be resolved, and the service should start without delay.