Release : 9.0.2
Component : UNIFIED INFRASTRUCTURE MGMT
The MarkLogic probe is a custom probe created via restmon using a schema/json file.
MarkLogic developes the NoSQL database and this probe provides monitoring for it.
The schema json is available in restmon_sample_schemas-1.12 which has marklogic_schema.json and is loaded in the Operator Console > Settings > restmon page.
Note: MarkLogicNode.json does not load.
Once loaded it creates a package in the archive and then a mcs template.
Setting up the template creates a profile with:
Connections Group 1
Database Details Group 1
Database Details Group 2
Database Details Group 3
Database Details Group 4
Forests Group 1
Metrics Group 1
Metrics Group 2
Severs Group 1
If the profile is configured correctly, it connects to the database and populates the profile with inventory items for configuration.
Once deployed only the restmon probe shows up as a probe at the robot.
The MarkLogic probe only shows up under installed packages .
In the archive it shows up with the name provided and a long character string after the name give.
It is to be deployed only via mcs and not directly.
At the robot the json file can be found at:
$\Nimsoft\probes\application\restmon\probe_schemas
For testing a rest call can be done via a browser.
For example to get the list of databases:
http://<sytem name>:port/manage/LATEST/databases?format=json
returns databases.json
sample:
{"database-default-list":{"meta":{"uri":"/manage/LATEST/databases",
"current-time":"2019-06-13T17:48:02.7572968-04:00",
"elapsed-time":{"units":"sec", "value":0.0187763}},
"list-items":{"list-count":{"units":"quantity", "value":10},
"list-item":[{"uriref":"/manage/LATEST/databases/App-Services",
"idref":"6929975297245519065", "nameref":"App-Services"},
{"uriref":"/manage/LATEST/databases/Documents", "idref":"3973080214977648989", "nameref":"Documents"},
{"uriref":"/manage/LATEST/databases/Extensions", "idref":"861095181410807649", "nameref":"Extensions"},
{"uriref":"/manage/LATEST/databases/Fab", "idref":"14613858095688383796", "nameref":"Fab"},
{"uriref":"/manage/LATEST/databases/Last-Login", "idref":"14236970057899539057", "nameref":"Last-Login"},
{"uriref":"/manage/LATEST/databases/Meters", "idref":"15406338172567144973", "nameref":"Meters"},
{"uriref":"/manage/LATEST/databases/Modules", "idref":"15367093762249271637", "nameref":"Modules"},
{"uriref":"/manage/LATEST/databases/Schemas", "idref":"5824046850443252275", "nameref":"Schemas"},
{"uriref":"/manage/LATEST/databases/Security", "idref":"9415669216441102616", "nameref":"Security"},
{"uriref":"/manage/LATEST/databases/Triggers", "idref":"11638969836904301574", "nameref":"Triggers"}]},
"related-views":{"related-view":[{"view-type":"root", "view-name":"default", "view-uri":"/manage/LATEST"},
{"view-type":"list", "view-name":"metrics", "view-uri":"/manage/LATEST/databases?view=metrics"},
{"view-type":"list", "view-name":"package", "view-uri":"/manage/LATEST/databases?view=package"},
{"view-type":"list", "view-name":"properties-schema", "view-uri":"/manage/LATEST/databases?view=properties-schema"},
{"view-type":"list", "view-name":"schema", "view-uri":"/manage/LATEST/databases?view=schema"},
{"view-type":"list", "view-name":"health", "view-uri":"/manage/v2?view=health&health-resource=database"},
{"view-type":"list", "view-name":"docs",
"view-uri":"http://docs.marklogic.com/REST/management/databases"}]}}}
Then this call to get the details for a database such as the App-Services database:
http://<sytem name>:port/manage/LATEST/databases/App-Services?view=status&format=json
this returns App-Services.json
sample:
{"database-status":{"id":"6929975297245519065",
"name":"App-Services",
"meta":{"uri":"/manage/LATEST/databases/App-Services?view=status",
"current-time":"2019-06-13T17:52:47.2062873-04:00",
"elapsed-time":{"units":"sec", "value":0.0248936}},
"relations":{"relation-group":[{"typeref":"databases",
"relation-count":{"units":"quantity", "value":1},
"relation":[{"uriref":"/manage/LATEST/databases/Security",
"roleref":"security", "idref":"9415669216441102616",
"nameref":"Security"}]}, {"typeref":"forests",
"relation-count":{"units":"quantity", "value":1},
"relation":[{"uriref":"/manage/LATEST/forests/App-Services", "idref":"16419162911945402702",
"nameref":"App-Services"}]}, {"typeref":"servers",
"relation-count":{"units":"quantity", "value":2},
"relation":[{"uriref":"/manage/LATEST/servers/HealthCheck?group-id=Default", "idref":"11076997515355309659",
"nameref":"HealthCheck"}, {"uriref":"/manage/LATEST/servers/Manage?group-id=Default", "idref":"282046132255437131",
"nameref":"Manage"}]}]}, "status-properties":{"enabled":{"units":"bool", "value":true},
"state":{"units":"enum", "value":"available"},
"indexing-state":{"units":"bool", "value":"false"},
"rebalancing-state":{"units":"bool", "value":"false"},
"forests-count":{"units":"quantity", "value":1},
"device-space":{"units":"MB", "value":"83199"},
"data-size":{"units":"MB", "value":8},
"large-data-size":{"units":"MB", "value":0},
"fast-data-size":{"units":"MB", "value":0},
"in-memory-size":{"units":"MB", "value":11},
"average-forest-size":{"units":"MB", "value":8},
"largest-forest-size":{"units":"MB", "value":8},
"least-remaining-space-forest":{"units":"MB", "value":83183},
"merge-count":{"units":"quantity", "value":0},
"total-merge-size":{"units":"MB", "value":0},
"reindex-count":{"units":"quantity", "value":0},
"backup-count":{"units":"quantity", "value":0},
"restore-count":{"units":"quantity", "value":0},
"min-capacity":{"units":"%", "value":"99.98077"},
"load-properties":{"total-load":{"units":"sec/sec", "value":0},
"load-detail":{"query-read-load":{"units":"sec/sec", "value":0},
"journal-write-load":{"units":"sec/sec", "value":0},
"save-write-load":{"units":"sec/sec", "value":0},
"merge-read-load":{"units":"sec/sec", "value":0},
"merge-write-load":{"units":"sec/sec", "value":0},
"backup-read-load":{"units":"sec/sec", "value":0},
"backup-write-load":{"units":"sec/sec", "value":0},
"restore-read-load":{"units":"sec/sec", "value":0},
"restore-write-load":{"units":"sec/sec", "value":0},
"large-read-load":{"units":"sec/sec", "value":0},
"large-write-load":{"units":"sec/sec", "value":0},
"database-replication-send-load":{"units":"sec/sec", "value":0},
"database-replication-receive-load":{"units":"sec/sec", "value":0},
"read-lock-wait-load":{"units":"sec/sec", "value":0},
"read-lock-hold-load":{"units":"sec/sec", "value":0},
"write-lock-wait-load":{"units":"sec/sec", "value":0},
"write-lock-hold-load":{"units":"sec/sec", "value":0},
"deadlock-wait-load":{"units":"sec/sec", "value":0}}},
"rate-properties":{"total-rate":{"units":"MB/sec", "value":0},
"rate-detail":{"query-read-rate":{"units":"MB/sec", "value":0},
"journal-write-rate":{"units":"MB/sec", "value":0},
"save-write-rate":{"units":"MB/sec", "value":0},
"merge-read-rate":{"units":"MB/sec", "value":0},
"merge-write-rate":{"units":"MB/sec", "value":0},
"backup-read-rate":{"units":"MB/sec", "value":0},
"backup-write-rate":{"units":"MB/sec", "value":0},
"restore-read-rate":{"units":"MB/sec", "value":0},
"restore-write-rate":{"units":"MB/sec", "value":0},
"large-read-rate":{"units":"MB/sec", "value":0},
"large-write-rate":{"units":"MB/sec", "value":0},
"database-replication-send-rate":{"units":"MB/sec", "value":0},
"database-replication-receive-rate":{"units":"MB/sec", "value":0},
"read-lock-rate":{"units":"MB/sec", "value":0},
"write-lock-rate":{"units":"MB/sec", "value":0},
"deadlock-rate":{"units":"MB/sec", "value":0}}},
"cache-properties":{"list-cache-hit-rate":{"units":"hits/sec", "value":0},
"list-cache-miss-rate":{"units":"misses/sec", "value":0},
"triple-cache-hit-rate":{"units":"hits/sec", "value":0},
"triple-cache-miss-rate":{"units":"misses/sec", "value":0},
"triple-value-cache-hit-rate":{"units":"hits/sec", "value":0},
"triple-value-cache-miss-rate":{"units":"misses/sec", "value":0},
"compressed-tree-cache-hit-rate":{"units":"hits/sec", "value":0},
"compressed-tree-cache-miss-rate":{"units":"misses/sec", "value":0},
"list-cache-ratio":{"units":"%", "value":"52"},
"compressed-tree-cache-ratio":{"units":"%", "value":"0"},
"large-binary-cache-hit-rate":{"units":"hits/sec", "value":0},
"large-binary-cache-miss-rate":{"units":"misses/sec", "value":0}},
"local-disk-failover":{"local-disk-failover-active":{"units":"bool", "value":false}},
"database-replication-status":{"database-replication-configured":{"units":"bool", "value":false}},
"shared-disk-failover":{"shared-disk-failover-active":{"units":"bool", "value":false}},
"flexible-replication-enabled":{"units":"bool", "value":false},
"cpf-enabled":{"units":"bool", "value":false}}, "related-views":{"related-view":
[{"view-type":"item", "view-name":"default", "view-uri":"/manage/LATEST/databases/App-Services"}]}}}
The return values can be used to verify what the probe reports show.
For troubleshooting all activity is written to the restmon log. At the robot restmon only has a raw configure available, and MarkLogic will be listed under Profiles. The profile should be modified only via mcs and not raw configure.
Available QoS obtained from marklogic_schema.json
QOS_HTTP_STATUS
QOS_ML_DB_LIB_MODULE_CACHE_HITS
QOS_ML_DB_LIB_MODULE_CACHE_MISSES
QOS_ML_DB_MAIN_MODULE_SEQ_CACHE_HITS
QOS_ML_DB_MAIN_MODULE_SEQ_CACHE_MISSES
QOS_ML_PROGRAM_CACHE_HITS
QOS_ML_DB_PROGRAM_CACHE_MISSES
QOS_ML_ENV_PROGRAM_CACHE_HITS
QOS_ML_ENV_PROGRAM_CACHE_MISSES
QOS_ML_EXPANDED_TREE_CACHE_HITS
QOS_ML_EXPANDED_TREE_CACHE_MISSES
QOS_ML_EXPANDED_TREE_CACHE_MISS_RATE
QOS_ML_EXPANDED_TREE_CACHE_HIT_RATE
QOS_ML_FS_LIB_MODULE_CACHE_HITS
QOS_ML_FS_LIB_MODULE_CACHE_MISSES
QOS_ML_FS_MAIN_MODULE_CACHE_HITS
QOS_ML_FS_MAIN_MODULE_CACHE_MISSES
QOS_ML_FS_PROGRAM_CACHE_HITS
QOS_ML_FS_PROGRAM_CACHE_MISSES
QOS_ML_QUEUE_SIZE
QOS_ML_REQUEST_RATE
QOS_ML_REQUEST_TIME
QOS_ML_THREADS
QOS_ML_HTTP_SERVER_THREADS
QOS_ML_ODBC_SERVER_THREADS
QOS_ML_TASK_SERVER_THREADS
QOS_ML_WEBDAV_SERVER_THREADS
QOS_ML_XDBC_SERVER_THREADS
QOS_ML_TOTAL_LOAD
QOS_ML_DB_BACKUP_READ_RATE
QOS_ML_DB_BACKUP_WRITE_RATE
QOS_ML_DB_REPLICATION_RECEIVE_RATE
QOS_ML_DB_REPLICATION_SEND_RATE
QOS_ML_DB_DEADLOCK_RATE
QOS_ML_DB_JOURNAL_WRITE_RATE
QOS_ML_DB_LARGE_READ_RATE
QOS_ML_DB_LARGE_WRITE_RATE
QOS_ML_DB_MERGE_READ_RATE
QOS_ML_DB_MERGE_WRITE_RATE
QOS_ML_DB_QUERY_READ_RATE
QOS_ML_DB_READ_LOCK_RATE
QOS_ML_DB_RESTORE_READ_RATE
QOS_ML_DB_RESTORE_WRITE_RATE
QOS_ML_DB_SAVE_WRITE_RATE
QOS_ML_DB_WRITE_LOCK_RATE
QOS_ML_DB_COMPRESSED_TREE_CACHE_HIT_RATE
QOS_ML_DB_COMPRESSED_TREE_CACHE_MISS_RATE
QOS_ML_DB_LARGE_BINARY_CACHE_HIT_RATE
QOS_ML_DB_LARGE_BINARY_CACHE_MISS_RATE
QOS_ML_DB_LIST_CACHE_HIT_RATE
QOS_ML_DB_LIST_CACHE_MISS_RATE
QOS_ML_DB_LIST_CACHE_RATIO
QOS_ML_DB_TRIPLE_CACHE_HIT_RATE
QOS_ML_DB_TRIPLE_CACHE_MISS_RATE
QOS_ML_DB_TRIPLE_VALUE_CACHE_HIT_RATE
QOS_ML_DB_TRIPLE_VALUE_CACHE_MISS_RATE
QOS_ML_DB_BACKUP_READ_LOAD
QOS_ML_DB_BACKUP_WRITE_LOAD
QOS_ML_DB_REPLICATION_RECEIVE_LOAD
QOS_ML_DB_REPLICATION_SEND_LOAD
QOS_ML_DB_DEADLOCK_WAIT_LOAD
QOS_ML_DB_JOURNAL_WRITE_LOAD
QOS_ML_DB_LARGE_READ_LOAD
QOS_ML_DB_LARGE_WRITE_LOAD
QOS_ML_DB_MERGE_READ_LOAD
QOS_ML_DB_MERGE_WRITE_LOAD
QOS_ML_DB_QUERY_READ_LOAD
QOS_ML_DB_READ_LOCK_HOLD_LOAD
QOS_ML_DB_READ_LOCK_WAIT_LOAD
QOS_ML_DB_RESTORE_READ_LOAD
QOS_ML_DB_RESTORE_WRITE_LOAD
QOS_ML_DB_SAVE_WRITE_LOAD
QOS_ML_DB_WRITE_LOCK_HOLD_LOAD
QOS_ML_DB_WRITE_LOCK_WAIT_LOAD
QOS_ML_DB_BACKUP_COUNT
QOS_ML_DB_CPF_ENABLED
QOS_ML_DB_DATA_SIZE
QOS_ML_DB_REPLICATION_STATUS
QOS_ML_DB_DEVICE_SPACE
QOS_ML_DB_ENABLED
QOS_ML_DB_FAST_DATA_SIZE
QOS_ML_DB_FLEXIBLE_REPLICATION_ENABLED
QOS_ML_DB_FORESTS_COUNT
QOS_ML_DB_IN_MEMORY_SIZE
QOS_ML_DB_INDEXING_STATE
QOS_ML_DB_LARGE_DATA_SIZE
QOS_ML_DB_LOCAL_DISK_FAILOVER
QOS_ML_DB_MERGE_COUNT
QOS_ML_DB_MIN_CAPACITY
QOS_ML_DB_REBALANCING_STATE
QOS_ML_DB_REINDEX_COUNT
QOS_ML_DB_RESTORE_COUNT
QOS_ML_DB_STATE
QOS_ML_DB_TOTAL_MERGE_SIZE
QOS_ML_FOREST_TRANSACTION_JOURNAL_SIZE
QOS_ML_SVR_ENABLED
QOS_ML_SVR_DISPLAY_LAST_LOGIN
QOS_ML_SVR_BACKLOG
QOS_ML_SVR_THREADS
QOS_ML_SVR_MAX_THREADS
QOS_ML_SVR_REQUEST_TIMEOUT
QOS_ML_SVR_KEEP_ALIVE_TIMEOUT
QOS_ML_SVR_SESSION_TIMEOUT
QOS_ML_SVR_STATIC_EXPIRES
QOS_ML_SVR_MAX_TIME_LIMIT
QOS_ML_SVR_DEFAULT_TIME_LIMIT
QOS_ML_SVR_MULTI_VERSION_CONCURRENCY_CONTROL
QOS_ML_SVR_AUTHENTICATION
QOS_ML_SVR_CONCURRENT_REQUEST_LIMIT
QOS_ML_SVR_OUTPUT_SGML_CHARACTER_ENTITIES
QOS_ML_SVR_TOTAL_REQUEST_RATE
QOS_ML_SVR_AVG_MISSES_EXPANDED_CACHE
QOS_ML_SVR_AVG_HITS_EXPANDED_CACHE
QOS_ML_SVR_TOTAL_REQUESTS
MarkLogic documentation:
https://docs.marklogic.com/REST/management
The doc pages can be used to better understand what each metric returns.
For example Backup Count QOS_ML_DB_BACKUP_COUNT would seem to indicate the number of times the database has been backed up. However the doc page shows:
backup-count
The maximum number of forests that are backing up. (meaning currently active backups).