Best practice recommendations on how to determine what needs to be cached, when using selective caching.

Products

CA Directory CA Security Command Center CA Data Protection (DataMinder) CA User Activity Reporting

Issue/Introduction

Description

This document provides best practice recommendations on how to determine what needs to be cached, when using selective caching.

Environment

Release: 8.1.983 or later
Component: ETRDIR

Resolution

Solution

Before you customize the DXcache index, use the query log to collect all searches submitted to a DSA. You can then analyze the log information, and use the most frequent LDAP search filters to populate the DXcache index.

Set Up the Query Log

To determine what attributes need to be added to the DXcache configuration perform the following for the required DSA:

Enable the query log by adding the following parameter in the logging configuration file:
```
    set query-log = "logs/$s_query.log";
```
Enable the logging of the returned attributes by adding the following parameter to the logging configuration file:
```
    set query-log-show-eis = true;
```
"eis" is the acronym for Entry Information Selection. This command tells CA Directory to log the attributes that were requested to be returned by the LDAP search.
Stop and restart the DSA.

The query log will be created and have the following filename structure:
```
    {dsaName}_query_ccyymmdd.log
```
With the query log active, execute all LDAP application features and functions to populate the query log with all the required information.

We recommend that you leave the query log running on the DSA for 7 days to check the frequency of each type of query. The frequency of each type of query may also determine if you cache the required attributes.

Analyze the Queries

Output from the query log with the above parameters set looks like the following:

    20070227.134252.209 2.3 SEARCH dn="o=DEMOCORP,c=AU" scope=subtree 
    filter=(&(sn=B*)(cn=A*)) eis=sn cn ou organizationalStatus ou 
    personalTitle postalCode

The two cache parameters that are configured from the contents of the query log are:

cache-index
cache-attrs

The cache-index attributes are those that make up the LDAP search filter. In the query log, the LDAP search filter can found after the "filter=" item.

The cache-attrs attributes are those that are found after "eis=" in the query log. The following table illustrates the relationship between the parameters and the log file components:

DXcache parameter	Query log item	Example
set cache-index	(&(sn=B)(cn=A))	set cache-index = sn, cn;
set cache-attrs	eis=sn cn ou organizationalStatus ou personalTitle postalCode	set cache-attrs = sn, cn, ou, organizationalStatus, ou, personalTitle, postalCode;

A Perl script is attached which processes a query log automatically and outputs the number of times each LDAP search filter has been used to this document (see query-count.zip). If you do not have enough RAM to cache all your attributes in the index, then this script can be used to highlight the LDAP searches that occur frequently.

Populate your cache index with the attributes that are searched for most frequently.

Before you enable the cache, define its parameters.

Set the Maximum Cache Size

Enabling the cache will not work unless you specify the maximum memory that the cache may use. The cache always uses the minimum amount, so this figure is simply to ensure that it has been properly dimensioned.

The cache is designed to be memory-resident, so if its size is 2GB but there is only 500MB of RAM, there has been a mis-configuration. Also note that you may have to allow 250MB (or more) for the operating system and any multiwrite queues that you may have configured.

To check the size of an existing cache, use the following DXconsole command:

    get cache;

For example, to configure a 1.5 GB cache on a 2 GB system, use the following command:

    set max-cache-size = 1500;

Set the Attributes to be Indexed

In order for the cache to efficiently process search requests, the attributes appearing in the search filters should be indexed. For example:

    set cache-index = commonName, surname;

The cache will hold all entries that contain one or more of these attributes. Entries that contain none of the attributes in this list will not be loaded into the cache. The choice of indexed attributes directly affects the number of entries in the cache.

Note: None of the attributes in the cache-index command can be marked as "not-searchable" or the cache will not load correctly.

The attributes to be returned by search requests have to be cached. The indexed attributes are already cached so we only need to specify additional attributes. For example:

    set cache-attrs = description;

Search requests that do not specifically request particular attributes to be returned are actually implicitly requesting the return of all attributes. To cater for this type of request, use

    set cache-attrs = all-attributes;

Caching all attributes requires more memory than caching only some attributes.

Activate DXcache

To activate the cache, uncomment out the section at the bottom of the DSA's initialization file found in the following folder:

Win32: %DXHOME%\config\servers
Unix/Linux: $DXHOME/config/servers

An example DXcache configuration is listed below:

    # cache configuration
    set max-cache-size = 100;
    set cache-index = commonName, surname;
    set cache-attrs = uid, rollNumber;
    set lookup-cache = true;

The cache is enabled with the command

    set lookup-cache = true;

Debugging DXcache

If the "max-cache-size" setting is too small for the amount of data being loaded, then there will be messages written to the DSA's alarm log indicating that the cache is filling up. Indicators start at the 60% mark, with further messages being written at the 70%, 80%, 90% and 100% marks.

Once the 100% level is reached, and the cache still needs to load more data, the DSA will disable the cache, generate an alarm message reading "Cache disabled", and revert to a normal data DSA.

If the cache does successfully load, but performance is not what is expected, then there are two debugging procedures that can be used to analyze DXcache.

They are:

Reviewing the runtime cache configuration
Activating SQL tracing to analyze the SQL calls being made to the database.

Reviewing the Runtime DXcache Configuration

To review the runtime DXcache configuration:

Start a DXconsole session and connect to the DSA's console port.

Type in the following command:

get cache;

If the cache is disabled, this will display the following information:

  Cache disabled
  cache-only = FALSE 
  cache-attrs = 
  cache-index =

If the cache is enabled and loaded, this command displays the following information: Cache enabled

  cache-max-size = 100000000 
  cache-only = FALSE 
  cache-attrs = all-attributes 
  cache-index = 
    aliasedObjectName surname commonName 
  cache-reverse =

  Number of Entries 1276 
  Memory used by cache: 1693392 out of 2343316 
    (Memory initially used by data only: 1455576)
  Memory leaked 0

  Number of Cache Hits 0 
  Number of Cache Misses 1 
  Number of Sequential Scans 0 
  Free lists: 
    EntryLists: 0 
    Constrained values: 0 
    Unconstrained values: 0 
    IndexList: 0

The amount of entries loaded and the total memory utilized will depend on what attributes you are indexing and loading.

To detect mis-configured DXcache DSA's, look for increasing:

Cache misses
Cache sequential scans
Low number of cache hits

If any of these are regularly increasing, then the DXcache configuration is not correctly tuned to manage the traffic that the DSA is receiving. Revisit the cache configuration and adjust where necessary.

Activate the SQL-log

The SQL log contains searches that are being passed to the database for execution. If DXcache is configured correctly, then you should not see any SQL searches being submitted to Ingres for the attributes that you are indexing. If there are, then the cache configuration needs to be refined.

To enable SQL level tracing for the DXcache DSA, enter the following command via a DXconsole session:

    set trace=sql;