Watch4Net, ViPR SRM: Import properties on frontend taking too long to run
search cancel

Watch4Net, ViPR SRM: Import properties on frontend taking too long to run


Article ID: 324755


Updated On:





The M&R Frontend runs a scheduled task called  import-properties , which creates an indexed database (property-store) of metrics and properties used during report generation.   This database is normally found in  /opt/APG/Databases/APG-Property-Store .  

As the number of metrics and properties increases, it takes longer to generate the property-store.  The conditional execution logic of the task, by default, will schedule the import-properties task to run once a day at midnight if the average time to generate is more than 1 hour (conditional execution logic can be changed in import-properties.task). 

This has a negative impact on the timeliness of data in reports.  If the property-store is recreated each day at midnight and takes 12 hours, a device added to collection on Tuesday morning will not appear in reports until Wednesday afternoon.

Dell EMC engineering recommends that the import-properties task can be reconfigured to run more quickly, so that changes to metrics and properties are reflected in reports sooner.


VMware Smart Assurance - Watch4Net/M&R


There are two steps in property-store generation, each can be optimized by parallel processing.
  1. Data extraction and fetching.  The metrics and properties from each database are exported and retrieved by the front end.
  2. Indexing.  The fetched data is indexed to create the property-store used by the frontend.
Optimizing data extraction and fetching by running import-properties on each backend
An import-properties task can be installed and run on each backend.  This will export the metrics and properties for the database into a file that can be fetched by the frontend, without the frontend needing to run a query for each database.
Install one property-Store package per backend
So, you ll have one property-store on the PBE and 4 property-store on each ABE
This is distribute the task execution in smaller/local chunks and FE will pull a pre-generated version instead of generating itself.
PBE install property-store Default
ABE install property-store apg1 install property-store apg2 install property-store apg3 install property-store apg4
On the ABE, it will ask you each time where is the backend associated to the property-store.
# install property-store apg4
Required dependencies, in processing order:
   [1]   java '8.0.51' v8.0.51
   [2]   module-manager '1.8u1' v1.8u1
   [3]   jdbc-drivers 'Default' v2.6
   [4] I property-store 'apg4' (none) => v2.1u1
> 3 not modified, 1 to install
> 7.5 MB space required / 89.7 GB available
? Enter the step to modify, 'yes' to accept them, or 'no' to cancel the operation [yes] > y
Starting installation of APG-Property-Store v2.1u1 from property-store-2.1u1-linux-x64...
* Gathering information...
* 'APG-Property-Store v2.1u1' will be registered with instance name 'apg4'.
* It will be installed in '/opt/APG/Databases/APG-Property-Store/apg4'.
* Unpacking files...
   [1] Backends/APG-Backend/apg1/conf/mysql.xml
   [2] Backends/APG-Backend/apg2/conf/mysql.xml
   [3] Backends/APG-Backend/apg3/conf/mysql.xml
   [4] Backends/APG-Backend/apg4/conf/mysql.xml
? Which file should the 'import-property.task' be configured with ? [4] > 4
* Installing files... 100%
* 42 files have been installed.
* Finalizing installation...
? Task file import-properties.task added by module property-store apg4. Enable it? (yes/no/All/None) [y] > y
Installation complete.
Each import-properties task should appear in the GUI and should run without errors.  The BE system task scheduler log file should have messages like the below:
[root]# tail scheduler-0-0.log
INFO     -- [2017-01-30 10:00:00 CST] -- : Task import-properties-Default -- Starting task import-properties-Default.
INFO     -- [2017-01-30 10:00:01 CST] -- : Task import-properties-Default -- Successful completion of task import-properties-Default with status 0.
The extracted data is in the following location on the BE system by default:
[root]# ll
total 4
drwxr-xr-x 3 apg apg 4096 Feb 16 16:11 _NONE_
-rw-r--r-- 1 apg apg    0 Jan 27 01:00 _NONE_.lock
The import-properties task running on the frontend does not require any changes, if the extracted data is found on the backend it will be fetched, with messages like the following in the FE system taskRun log file:
170216 01:02:38 : Feb 16, 2017 1:02:38 AM com.watch4net.propstore.apg.tasks.FetchPropertiesTask compute
170216 01:02:38 : INFO: Fetching data table 'data_property' from partition 'APG-apg1' into '/opt/APG/Databases/APG-Property-Store/Default/tmp/DB/APG-DB-nsm123vp141b-1'...
170216 01:02:39 : Feb 16, 2017 1:02:39 AM com.watch4net.propstore.apg.tasks.FetchPropertiesTask completed
Improvements to the frontend task run time should be visible in the GUI under  Scheduled Tasks .
Optimizing Indexing
After the metrics and properties for each backend are fetched they are indexed and written out to the APG-Property-Store directory.  The size of this directory can be obtained as follows:
[root2@nsmdenvp138b DB]# pwd
[root2@nsmdenvp138b DB]# du -m
17009   ./1487206984103
16996   ./1487120564721
34004   .
As a rule, this directory can be split up into smaller directories, each being around 5gb in size.  In the procedure below, it is assumed that the monolithic cache DB is being split into 6: (DB1..DB6) 
  1. Disable the import-properties tasks on the frontend, stop the task if it is running. 
  2. Before starting procedure: save copy of resources table from master database on PBE
/opt/APG/Databases/MySQL/Default/bin/mysqldump  defaults-file=/opt/APG/Databases/MySQL/Default/my.cnf --user=root -p master resources > /root/resources.sql
  1. On the PBE, run update  for each APG database, changing the  setting:cachegrp  so that instead of being DB in every case, it uses a value from (DB1..DB6). After running the commands to update the resources, the import-properties tasks should be run for each backend. 
To get list of resources:
/opt/APG/bin/ list | grep APG
    dba/APG-DB-nsm123vp141b-1 ,
Update cachegrp for each APG DB:
/opt/APG/bin/ update dba/APG-DB "{'settings':{'cachegrp':'DB1'}}"
/opt/APG/bin/ update dba/APG-DB-nsm123vp141b-1 "{'settings':{'cachegrp':'DB2'}}"
4.Once the import-properties tasks have completed, on the frontend:
a.Stop the tomcat service ( service tomcat stop)
b.Delete the APG-Property-Store DB by removing the directory  /opt/APG/Databases/APG-Property-Store/Default/data/DB 
c.Run the import-properties task from the GUI.  This may take several hours, so there will be an extended pause in the procedure
d.Check that the directory  /opt/APG/Databases/APG-Property-Store/Default/data  has subdirectories DB1..DB6.
e.Start the tomcat service ( service tomcat start)
f.Login with the Web GUI and confirm that reports work as before.
5.If there is a need to back out
a.Stop tomcat on frontend
b.Restore the resources table from master database from the saved version.
/opt/APG/Databases/MySQL/Default/bin/mysql --defaults-file=/opt/APG/Databases/MySQL/Default/my.cnf --user=root -p master < /root/resources.sql
c.Delete the (DB1..DB6) directories and run the import-properties task, that should recreate the DB directory
d.Start tomcat on front end and check reports
6.Enable the import-properties tasks on front end.  Check the time to run.