NSX Network Detection and Response - Lastline utility tools
Article ID: 323954
Updated On:
Products
Issue/Introduction
All the appliances have some python scripts to configure or test the appliances.
Here are the most common ones along with their usage and options.
In this article:
1. lastline_register
Available on: All appliances
Use case: To register new appliances or change current configurations (See tool available options).
2. lastline_test_appliance
Available on: All appliances
Use case: To check the general health status.
3. lastline_apply_config
Available on: All appliances
Use case: to re-trigger configurations.
4. lastline_setup
Available on: All appliances
Use case: To change current configurations (See tool available options).
5. get_appliance_status.py
Available on: All appliances
Use case: To see information like license information, capabilities, details about the appliance, integration components and system status, last check in.
6. lastline_check_analysis_submission_load
Available on: Manager
Use case: To get submission details when troubleshooting analysis related issues.
7. lastline_distribution_upgrade
Available on: All appliances
Use case: To upgrade Ubuntu distribution.
8. lastline_diagnostic_bundle.py
Available on: All appliances
Use case: Useful to troubleshoot appliances with the support channel closed, this bundle will gather accounting information, monitoring logs, logs, appliance config, child appliances general details.
When pointing to a config file we can modify the files we gather. See tool available options to get additional details.
9. lastline_download_engine_data (update-llama-images.py)
Available on: Manager, Engine.
Use case: To download and update llama images to the latest or specified version and language.
10. lastline_prune_analysis_backlog
Available on: Manager
Use case: To prune pending analysis tasks. Use only as a last resort when troubleshooting queue issues.
11. lastline_prune_docker_images
Available on: All appliances
Use case: To troubleshoot or manage docker images.
12. lastline_worker_registration (Deprecated, see instead lastline_register --force-register-all-workers)
Available on: Engine
Use case: When a manager is replaced, some workers might need to be re-registered against the new manager, if that is the case we will see workers not checking after of running lastline_test_appliance on the manager.
13. get_tres_bundle.py
Available on: Manager
Use case: Needed and useful when investigating analysis issues like false positives/negatives and delay.
14. get_backup_status.py
Available on: All appliances (However, only the manager will return backup information when existing)
Use case: Useful while troubleshooting backup issues.
Resolution
1. lastline_register
Usage:
lastline_register
Options:
-h, --help show this help message and exit
--log=LOG_FILENAME Log to this file
-d, --debug Enable debug-mode
-q QUIET, --quiet=QUIET
Disable user-interaction and take answers from the provided answer configuration file
--no-lock Run lastline_register without acquiring a lock on /var/run/lastline_apply_config.lock
--customer=CUSTOMER Run registration on behalf of specified customer (for internal use only)
-C, --skip-apply Do not apply configuration
-B, --skip-reboot Do not reboot after configuration, even if necessary
--skip-all-tests Do not run any appliance tests. This options must not be used unless directed so by Lastline support.
--skip-hardware-tests. Do not run appliance hardware tests
--no-dist-upgrade Do not run dist-upgrade
-e, --expert-mode Run tool in 'expert-mode', asking for more configuration options
--no-ntp-sync Disable syncing with NTP specified during registration
-s, --skip-tests Legacy option. See --skip-all-tests or --skip-hardware-tests
--license-bundle=LICENSE_BUNDLE
Path to the license bundle needed for registration
--threat-intelligence-bundle=THREAT_INTELLIGENCE_BUNDLE
Path to the threat intelligence bundle needed for registration
--change-local-fqdn=CHANGE_LOCAL_FQDN
Change the local FQDN of this appliance
--change-active-manager-fqdn=CHANGE_ACTIVE_MANAGER_FQDN
Change the FQDN of the active manager this appliance depends from (do not use on active manager, analyst, pinbox)
--change-active-manager-ip=CHANGE_ACTIVE_MANAGER_IP
Change the ip of the active manager this appliance depends from (do not use on active manager,
analyst, pinbox)
-k ACCESS_KEY, --license-key=ACCESS_KEY
License key
--engine-key=ENGINE_KEY
Engine license key
2. lastline_test_appliance
Usage
lastline_test_appliance [options]
to run all tests, or
lastline_test_appliance [options] [category[:name] [category[:name] ...]]
to run a specific test, a set of tests, or all tests of the selected
categories, or
lastline_test_appliance [options] --tags <tag1>,<tag2> <category[:name] ...>
to run tests for a set of tags (optionally also by giving a name), or
lastline_test_appliance [options] ?
to print a list of categories/tests available
positional arguments:
CHECK_NAMES
optional arguments:
-h, --help Show this help message and exit
--config-file CONFIG_FILE
-p HTTP_PROXY, --http-proxy HTTP_PROXY
Use this HTTP-proxy for outgoing connections; use '-' to override default configuration to 'no proxy'
--log LOG_FILENAME Log to this file
-v, --verbose Enable verbose logging
-q, --quiet Disable most logging
-d, --debug Enable debug-mode
--no-fix Disable auto-fixing (no effect, present for legacy purposes).
--auto-fix Automatically fix checks if possible.
--no-upload-results Don't report results to Lastline backend
--no-verbose-reporting. If set, only important (started, errors/warnings, and completed) messages are reported
--max-verbosity MAX_VERBOSITY
Maximum verbosity of tests to run (for legacy purposed only)
--assume-yes Assume 'yes' to all questions (for legacy purposes only)
--lock-timeout LOCK_TIMEOUT
Amount of time (in seconds) to allow for acquiring the configuration lock. A negative value is interpreted as wait forever. Default is 60 seconds.
--no-lock Run lastline_test_appliance without acquiring a lock
--tags CHECK_TAGS Comma-separated list of check tags to run
--disable-tags DISABLE_CHECK_TAGS
Comma-separated list of check tags to not run
--no-default-disable-tags. Do not disable tags that are normally disabled by default
3. lastline_apply_config
Usage:
lastline_apply_config [-h] [-d] [-f] [-o] [-g] [-l] [-n] [--no-lock]
[--lock-timeout LOCK_TIMEOUT]
[--skip-hardware-checks]
[--skip-pre-puppet-db-migrations]
[--skip-kernel-modules-check]
[puppet_flags [puppet_flags ...]]
Positional arguments:
puppet_flags
Additional puppet flags.
Optional arguments:
-h, --help Show this help message and exit
-d Debug mode. Sets -f and -o.
-f Do not filter out uniteresting lines (e.g. deprecation warnings) from output.
-o Log output to standard output (possibly in addition to the log file).
-g Generate puppet dependency graph.
-l Do not log output to file. Implies -o.
-n Simulate (run puppet with --noop).
--no-lock Do not acquire lock on /var/run/lastline_apply_config.lock. This is intended only for usage of when a parent process that calls this program after already acquiring a lock on /var/run/lastline_apply_config.lock
--lock-timeout LOCK_TIMEOUT
If a lock on /var/run/lastline_apply_config.lock cannot be acquired within 3 seconds, a second attempt
to acquire the lock on /var/run/lastline_apply_config.lock will be made with a timeout in seconds
specified by this option (Default of 60).
--skip-hardware-checks
Disable checking if hardware is supported before applying configuration. For Lastline internal use only.
--skip-pre-puppet-db-migrations
Disable running DB migrations before applying the puppet catalog
--skip-kernel-modules-check. Disable checking if necessary kernel module packages exist for the currently installed kernels. For
Lastline internal use only.
4. lastline_setup
Usage: lastline_setup [-h] [--lock-timeout LOCK_TIMEOUT]
optional arguments:
-h, --help Show this help message and exit
--lock-timeout LOCK_TIMEOUT
Amount of time (in seconds) to allow for acquiring the configuration lock. Default is 0 seconds.
To get details about the available configurations using this tool see:
https://user.lastline.com/lastline-pdf-opsguide-manuals/Administration_Operations_Guide.html#setupoptions
5. get_appliance_status.py
Usage:
Get an appliance's status
-----------------------------------------
The full status of the appliance is dumped in JSON format to standard output.
* Run based on a configuration file to get information about a specific appliance:
get_appliance_status.py -c CONFIG --appliance-uuid UUID
For a sample configuration INI file, see papi_client.ini.template
* Run on the appliance to get information about itself:
get_appliance_status.py -u USERNAME -p PASSWORD
In this case, the appliance UUID as well as configuration for accessing the
API server will be fetched from standard locations on the appliance itself.
Optional arguments:
-h, --help Show this help message and exit
-c CONFIG, --config CONFIG
Configuration file name
--section SECTION Section of configuration file to read from
--appliance-uuid APPLIANCE_UUID
Unique identifier of appliance on which we want information
--username USERNAME Authenticate to API with this username
--password PASSWORD Authenticate to API with this password
To see details about the usage of the papi_client see:
https://user.lastline.com/papi-doc/api/html/intel/overview.html
6. lastline_check_analysis_submission_load
Usage: lastline_check_analysis_submission_load [-h] [-c CONFIG_FILE]
[--csv WRITE_TO_CSV] -s START_TS
[-e END_TS]
[--customer CUSTOMER_FILTER]
[--license LICENSE_FILTER]
[-b {day,hour,15-minute}]
[--all-licenses] [--all-clients]
[--all-mime-types] [--by-mime-type]
[--by-connection-protocol]
[--by-connection-server-ip]
[--window-size-hours WINDOW_SIZE_HOURS]
[--window-size-minutes WINDOW_SIZE_MINUTES]
[--ignore-duration-task-time-hours IGNORE_DURATION_TASK_TIME_HOURS]
[--include-cached-tasks-in-duration]
[--duration-percentile DURATION_PERCENTILE]
[--sort {total,new_tasks,cached_tasks}]
[--lock TASK_RUNNER_LOCK]
[--raise-on-lock-busy]
[--catch-exceptions]
[--no catch-exceptions]
[--email-to TASK_RUNNER_EMAIL_TO]
[--email-ts-file TASK_RUNNER_EMAIL_TS_FILE]
[--max-email-frequency TASK_RUNNER_MAX_EMAIL_FREQUENCY]
[--error-email] [--no-error-email]
[--send-log-on-error]
[--no-send-log-on-error]
[--retry-file TASK_RUNNER_RETRY_FILE]
[--retry-task RETRY]
[--error-email-ts-file TASK_RUNNER_ERROR_EMAIL_TS_FILE]
[--max-error-email-frequency TASK_RUNNER_MAX_ERROR_EMAIL_FREQ]
[--non-fatal-email]
[--no-non-fatal-email]
[--non-fatal-email-ts-file TASK_RUNNER_NON_FATAL_EMAIL_TS_FILE]
[--max-non-fatal-email-frequency TASK_RUNNER_MAX_NON_FATAL_EMAIL_FREQ]
[--llmonitoring-source TASK_RUNNER_LLMONITORING_SOURCE]
[--log-dir SIMPLE_LOGGER_LOGGER_DIR]
[--logger-name SIMPLE_LOGGER_LOGGER_NAME]
[--console-log-level {none,debug,info,warning,error,critical}]
[--file-log-level {none,debug,info,warning,error,critical}]
[--stdout-log-level {none,debug,info,warning,error,critical}]
[--console-log-format {long_process_thread,long_thread,short,json,long,custom}]
[--file-log-format {long_process_thread,long_thread,short,json,long,custom}]
[--custom-log-format SIMPLE_LOGGER_CUSTOM_LOG_FORMAT]
[--log-rotation-files ROTATION_FILES]
[--log-rotation-max-size ROTATION_MAX_SIZE]
[--log-rotation-error-files ROTATION_ERROR_FILES]
[--log-rotation-error-max-size ROTATION_ERROR_MAX_SIZE]
Optional arguments:
-h, --help Show this help message and exit
-c CONFIG_FILE, --config-file CONFIG_FILE
Read config from here
--csv WRITE_TO_CSV Write results to this CSV file (in addition to stdout)
-s START_TS, --start-ts START_TS
Date-range start (YY:MM:DD [HH:MM:SS[:f]])
-e END_TS, --end-ts END_TS
Date-range end (YY:MM:DD [HH:MM:SS[:f]], default is UTC now)
--customer CUSTOMER_FILTER
Only analyze submissions for this customer (default is all)
--license LICENSE_FILTER. Only analyze submissions for this license (default is all)
-b {day,hour,15-minute}, --bucketize {day,hour,15-minute}
Group output into time windows of this size (default is 'hour')
--all-licenses Do not distinguish between different licenses
--all-clients Do not distinguish between different client IPs
--all-mime-types Deprecated; see --by-mime-type
--by-mime-type Distinguish between mime-types; note that calculating usage per mime-type can be costly on large
installations, so using this parameter is not recommended unless a detailed analysis is needed
--by-connection-protocol. Distinguish by connection protocol reported by the client (if the client reports connection metadata)
--by-connection-server-ip Distinguish by connection server IP reported by the client (if the client reports connection metadata)
--window-size-hours WINDOW_SIZE_HOURS
Deprecated; use --window-size-minutes instead
--window-size-minutes WINDOW_SIZE_MINUTES
Operate in buckets of submissions of this many minutes (decrease for large amounts of data);
default is 15
--ignore-duration-task-time-hours IGNORE_DURATION_TASK_TIME_HOURS
Internal tuning parameter: exclude submissions duration if timing is beyond this many hours; this allows excluding analysis results that were updated at a later time; default is 12
--include-cached-tasks-in-duration
When computing submission duration, include data from fully-cached tasks in the analysis; this gives a more complete picture of duration, but may be skewed if many submissions result in cached results
--duration-percentile DURATION_PERCENTILE
Report submission duration as average, maximum, and this percentile; default is 95
--sort {total,new_tasks,cached_tasks}
The column on which to sort the bucket stats (default is 'total' submissions)
TaskRunner Options:
--lock TASK_RUNNER_LOCK
Use these lockfiles for UNIX advisory locking. Must be provided as a comma-separated string.
--raise-on-lock-busy Raise an exception if the lock file is busy
--catch-exceptions Catch all exceptions: turn this on if you want uncaught exceptions to be logged. This is also required for any error email to be ever sent.
--no catch-exceptions Do not catch all exceptions.
--email-to TASK_RUNNER_EMAIL_TO
Send emails to destination address: overrides default from configuration file.
--email-ts-file TASK_RUNNER_EMAIL_TS_FILE
Use this file to store the timestamp of the last email sent, to implement --max-email-frequency.
--max-email-frequency TASK_RUNNER_MAX_EMAIL_FREQUENCY
Restrict the send_email method to send emails at most so often. This is an interval in minutes. Requires
--email-ts-file
--error-email Send an error email if there is an uncaught exception.
Implies --catch-exceptions
--no-error-email Do not send error emails.
--send-log-on-error Include tail of log file in error emails (convenient, but can leak some information)
--no-send-log-on-error Do not include tail of log file in error emails (convenient, but can leak some information)
--retry-file TASK_RUNNER_RETRY_FILE
Store retry count here.
--retry-task RETRY Only send error email if task has failed RETRY consecutive times. Requires --retry-file
--error-email-ts-file TASK_RUNNER_ERROR_EMAIL_TS_FILE
Use this file to store the timestamp of the last error email sent, to implement
--max-error-email-frequency.
--max-error-email-frequency TASK_RUNNER_MAX_ERROR_EMAIL_FREQ
Restrict the --error-email option to send emails at most so often. This is an interval in minutes.
Requires --error-email-ts-file and --error-email
--non-fatal-email Send an error email on non-fatal errors (task_runner.critical() invocations)
--no-non-fatal-email Do not send email on non-fatal errors
--non-fatal-email-ts-file TASK_RUNNER_NON_FATAL_EMAIL_TS_FILE
Use this file to store the timestamp of the last non fatal error email sent, to implement
--max-non-fatal-email-frequency.
--max-non-fatal-email-frequency TASK_RUNNER_MAX_NON_FATAL_EMAIL_FREQ
Restrict the --non-fatal-email option to send emails at most so often. This is an interval in minutes.
Requires --non-fatal-email-ts-file and --non-fatal- email
--llmonitoring-source TASK_RUNNER_LLMONITORING_SOURCE
If there is an uncaught exception, send an exception log message through llmonitoring using the given
source. If llmonitoring.reporting was not initialized, use the default config file llmonitoring.ini in
/etc/lastline. This option implies --catch-exceptions.
SimpleLogger Options:
--log-dir SIMPLE_LOGGER_LOGGER_DIR
Directory for storing log files
--logger-name SIMPLE_LOGGER_LOGGER_NAME
Name of default logger (and base name of default log file)
--console-log-level {none,debug,info,warning,error,critical}
Log to console from this severity up (one of debug, info, warning, error, fatal, none)
--file-log-level {none,debug,info,warning,error,critical}
Log to file from this severity up (one of debug, info, warning, error, fatal, none)
--stdout-log-level {none,debug,info,warning,error,critical}
Log to stdout up to and including this severity.
Higher severities will go to stderr(one of debug, info, warning, error, fatal, none)
--console-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to console using selected format (one of short, long)
--file-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to file using selected format (one of short, long)
--custom-log-format SIMPLE_LOGGER_CUSTOM_LOG_FORMAT
Custom log format to be used when --<console/file>-log-format=custom
--log-rotation-files ROTATION_FILES
Number of log-rotated log files to store
--log-rotation-max-size ROTATION_MAX_SIZE
Size (in bytes) of log before being log-rotated
--log-rotation-error-files ROTATION_ERROR_FILES
Number of log-rotated error-log files to store
--log-rotation-error-max-size ROTATION_ERROR_MAX_SIZE
Size (in bytes) of error-log before being log-rotated
7. lastline_distribution_upgrade
Usage: lastline_distribution_upgrade [-h] [--lock TASK_RUNNER_LOCK]
[--raise-on-lock-busy]
[--catch-exceptions]
[--no catch-exceptions]
[--email-to TASK_RUNNER_EMAIL_TO]
[--email-ts-file TASK_RUNNER_EMAIL_TS_FILE]
[--max-email-frequency TASK_RUNNER_MAX_EMAIL_FREQUENCY]
[--error-email] [--no-error-email]
[--send-log-on-error]
[--no-send-log-on-error]
[--retry-file TASK_RUNNER_RETRY_FILE]
[--retry-task RETRY]
[--error-email-ts-file TASK_RUNNER_ERROR_EMAIL_TS_FILE]
[--max-error-email-frequency TASK_RUNNER_MAX_ERROR_EMAIL_FREQ]
[--non-fatal-email]
[--no-non-fatal-email]
[--non-fatal-email-ts-file TASK_RUNNER_NON_FATAL_EMAIL_TS_FILE]
[--max-non-fatal-email-frequency TASK_RUNNER_MAX_NON_FATAL_EMAIL_FREQ]
[--llmonitoring-source TASK_RUNNER_LLMONITORING_SOURCE]
[--log-dir SIMPLE_LOGGER_LOGGER_DIR]
[--logger-name SIMPLE_LOGGER_LOGGER_NAME]
[--console-log-level {none,debug,info,warning,error,critical}]
[--file-log-level {none,debug,info,warning,error,critical}]
[--stdout-log-level {none,debug,info,warning,error,critical}]
[--console-log-format {long_process_thread,long_thread,short,json,long,custom}]
[--file-log-format {long_process_thread,long_thread,short,json,long,custom}]
[--custom-log-format SIMPLE_LOGGER_CUSTOM_LOG_FORMAT]
[--log-rotation-files ROTATION_FILES]
[--log-rotation-max-size ROTATION_MAX_SIZE]
[--log-rotation-error-files ROTATION_ERROR_FILES]
[--log-rotation-error-max-size ROTATION_ERROR_MAX_SIZE]
[--config CONFIG] [--skip-module-update]
[--module-version MODULE_VERSION]
[--skip-instruction SKIP_INSTRUCTION]
[--list-steps | --resume-from RESUME_FROM | --run-step RUN_STEP | --revert-from REVERT_FROM | --revert- step REVERT_STEP]
Optional arguments:
-h, --help Show this help message and exit
--config CONFIG Configuration file
--skip-module-update Do not update the distribution upgrade module on startup
--module-version MODULE_VERSION
Version of the upgrade module to fetch when self- updating
--skip-instruction SKIP_INSTRUCTION
Skip over requiring the user to acknowledge instructions and information about running the upgrade.
--list-steps List all steps in the order they're meant to be executed in
--resume-from RESUME_FROM
Run steps starting from a step specified by this argument.
--run-step RUN_STEP Run a specific step.
--revert-from REVERT_FROM
Revert steps starting from a step specified by this argument
--revert-step REVERT_STEP Revert a specific step.
TaskRunner Options:
--lock TASK_RUNNER_LOCK
Use these lockfiles for UNIX advisory locking. Must be provided as a comma-separated string.
--raise-on-lock-busy Raise an exception if the lock file is busy
--catch-exceptions Catch all exceptions: turn this on if you want uncaught exceptions to be logged. This is also required for any error email to be ever sent.
--no catch-exceptions Do not catch all exceptions.
--email-to TASK_RUNNER_EMAIL_TO
Send emails to destination address: overrides default from configuration file.
--email-ts-file TASK_RUNNER_EMAIL_TS_FILE
Use this file to store the timestamp of the last email sent, to implement --max-email-frequency.
--max-email-frequency TASK_RUNNER_MAX_EMAIL_FREQUENCY
Restrict the send_email method to send emails at most so often. This is an interval in minutes.
Requires --email-ts-file
--error-email Send an error email if there is an uncaught exception.
Implies --catch-exceptions
--no-error-email Do not send error emails.
--send-log-on-error Include tail of log file in error emails (convenient, but can leak some information)
--no-send-log-on-error Do not include tail of log file in error emails (convenient, but can leak some information)
--retry-file TASK_RUNNER_RETRY_FILE
Store retry count here.
--retry-task RETRY Only send error email if task has failed RETRY consecutive times. Requires --retry-file
--error-email-ts-file TASK_RUNNER_ERROR_EMAIL_TS_FILE
Use this file to store the timestamp of the last error email sent, to implement
--max-error-email-frequency.
--max-error-email-frequency TASK_RUNNER_MAX_ERROR_EMAIL_FREQ
Restrict the --error-email option to send emails at most so often. This is an interval in minutes.
Requires --error-email-ts-file and --error-email
--non-fatal-email Send an error email on non-fatal errors (task_runner.critical() invocations)
--no-non-fatal-email Do not send email on non-fatal errors
--non-fatal-email-ts-file TASK_RUNNER_NON_FATAL_EMAIL_TS_FILE
Use this file to store the timestamp of the last non fatal error email sent, to implement
--max-non-fatal-email-frequency.
--max-non-fatal-email-frequency TASK_RUNNER_MAX_NON_FATAL_EMAIL_FREQ
Restrict the --non-fatal-email option to send emails at most so often. This is an interval in minutes.
Requires --non-fatal-email-ts-file and --non-fatal-email
--llmonitoring-source TASK_RUNNER_LLMONITORING_SOURCE
If there is an uncaught exception, send an exception log message through llmonitoring using the given source. If llmonitoring.reporting was not initialized, use the default config file llmonitoring.ini in
/etc/lastline. This option implies --catch-exceptions.
SimpleLogger Options:
--log-dir SIMPLE_LOGGER_LOGGER_DIR
Directory for storing log files
--logger-name SIMPLE_LOGGER_LOGGER_NAME
Name of default logger (and base name of default log file)
--console-log-level {none,debug,info,warning,error,critical}
Log to console from this severity up (one of debug, info, warning, error, fatal, none)
--file-log-level {none,debug,info,warning,error,critical}
Log to file from this severity up (one of debug, info, warning, error, fatal, none)
--stdout-log-level {none,debug,info,warning,error,critical}
Log to stdout up to and including this severity. Higher severities will go to stderr(one of debug, info,
warning, error, fatal, none)
--console-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to console using selected format (one of short, long)
--file-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to file using selected format (one of short, long)
--custom-log-format SIMPLE_LOGGER_CUSTOM_LOG_FORMAT
Custom log format to be used when --<console/file>-log-format=custom
--log-rotation-files ROTATION_FILES
Number of log-rotated log files to store
--log-rotation-max-size ROTATION_MAX_SIZE
Size (in bytes) of log before being log-rotated
--log-rotation-error-files ROTATION_ERROR_FILES
Number of log-rotated error-log files to store
--log-rotation-error-max-size ROTATION_ERROR_MAX_SIZE
Size (in bytes) of error-log before being log-rotated
8. lastline_diagnostic_bundle.py
Usage:
lastline_diagnostic_bundle.py [options]
Generate a diagnostic bundle containing useful information to be used for trouble-shooting.
The input .ini config file must have the following structure:
[diagnostic_bundle]
# write the bundle file here, overridden by cmd line if provided
bundle_file = /var/lib/lastline/diagnostic_bundle.zip
# comma-separated list of files
previct_config_exclude_files = analyst_repo_password,sensor_repo_password
# comma-separated list of files
appliance_config_exclude_files = secrets.yaml
# comma-separated list of log directories to dump
log_dirs_to_dump = nginx,uwsgi
# get monitoring logs from the last n hours
monitoring_logs_hours = 3
[papi]
# overridden by cmd line if provided
username = your_user@your_site.com
# overridden by cmd line if provided
password = ***********
Optional arguments:
-h, --help Show this help message and exit
-c CONFIG_FILE, --config-file CONFIG_FILE
Read config from here
--username USERNAME Use this username for account-based authentication to the Manager. Required on slave appliances.
--password PASSWORD Use this password for account-based authentication to the Manager. Required on slave appliances.
WARNING: It is not recommended to use this option as it will show the password in the process list
--bundle-file BUNDLE_FILE Write the resulting bundle to this file
--current-appliance-only. Only get appliance management information about the current appliance, instead of all the appliances
managed by the customer
--skip-ssl Skip ssl verification when sending requests to gather information
--log LOG_FILENAME Log here
-v, --verbose Enable verbose logging
-q, --quiet Disable most logging
-d, --debug Enable debug-mode
9. lastline_download_engine_data (update-llama-images.py)
Identical to update-llama-images.py
Usage: lastline_download_engine_data [options]
Options:
-h, --help Show this help message and exit
-c CONFIG_FILE, --config-file=CONFIG_FILE
Specify the configuration file
-r REVISION, --revision=REVISION
Specify the Llama images revision to download
-i IMAGE_SET, --image-set=IMAGE_SET
Specify set of the Llama images to download
--additional-image-tag=ADDITIONAL_IMAGE_TAG
Specify locale of additional llama image tag to download
-n, --no-mark-current Only download the revision, do not mark it as the current image to use
-k, --insecure Disable SSL certificate validation
-f, --force-check Force MD5 check
--retries-without-progress=RETRIES_WITHOUT_PROGRESS
Number of attempts to do in case no progress is achieved at the previous attempt
--retries-with-progress=RETRIES_WITH_PROGRESS
Number of attempts to do in case progress is done at the previous attempt
--no-cleanup Do not cleanup old llama images revisions
--no-stamp Do not create stamp indicating that llama images are successfully installed
--no-download Do not attempt to download llama images; instead just check the validity of the revision specified that
should already be installed.
--images-dir=IMAGES_DIR Override images_directory from config
--no-stop-llama Override stop_llama from config to ensure that it is not stopped
--cdn Enforce the use of CDN servers, regardless of the configuration file setting
--no-cdn Disallow the use of CDN servers, regardless of the configuration file setting
--debug-zsync-requests Enable verbose logging of zsync requests
TaskRunner Options:
--lock=TASK_RUNNER_LOCK
Use these lockfiles for UNIX advisory locking. Must be provided as a comma-separated string.
--raise-on-lock-busy Raise an exception if the lock file is busy
--catch-exceptions Catch all exceptions: turn this on if you want uncaught exceptions to be logged. This is also required for
any error email to be ever sent.
--no catch-exceptions Do not catch all exceptions.
--email-to=TASK_RUNNER_EMAIL_TO
Send emails to destination address: overrides default from configuration file.
--email-ts-file=TASK_RUNNER_EMAIL_TS_FILE
Use this file to store the timestamp of the last email sent, to implement --max-email-frequency.
--max-email-frequency=TASK_RUNNER_MAX_EMAIL_FREQUENCY
Restrict the send_email method to send emails at most so often. This is an interval in minutes.
Requires --email-ts-file
--error-email Send an error email if there is an uncaught exception.
Implies --catch-exceptions
--no-error-email Do not send error emails.
--send-log-on-error Include tail of log file in error emails (convenient, but can leak some information)
--no-send-log-on-error Do not include tail of log file in error emails (convenient, but can leak some information)
--retry-file=TASK_RUNNER_RETRY_FILE
Store retry count here.
--retry-task=RETRY Only send error email if task has failed RETRY consecutive times. Requires --retry-file
--error-email-ts-file=TASK_RUNNER_ERROR_EMAIL_TS_FILE
Use this file to store the timestamp of the last error email sent, to implement
--max-error-email-frequency.
--max-error-email-frequency=TASK_RUNNER_MAX_ERROR_EMAIL_FREQ
Restrict the --error-email option to send emails at most so often. This is an interval in minutes.
Requires --error-email-ts-file and --error-email
--non-fatal-email Send an error email on non-fatal errors (task_runner.critical() invocations)
--no-non-fatal-email Do not send email on non-fatal errors
--non-fatal-email-ts-file=TASK_RUNNER_NON_FATAL_EMAIL_TS_FILE
Use this file to store the timestamp of the last non fatal error email sent, to implement
--max-non-fatal-email-frequency.
--max-non-fatal-email-frequency=TASK_RUNNER_MAX_NON_FATAL_EMAIL_FREQ
Restrict the --non-fatal-email option to send emails at most so often. This is an interval in minutes.
Requires --non-fatal-email-ts-file and --non-fatal-email
--llmonitoring-source=TASK_RUNNER_LLMONITORING_SOURCE
If there is an uncaught exception, send an exception log message through llmonitoring using the given
source. If llmonitoring.reporting was not initialized, use the default config file llmonitoring.ini in
/etc/lastline. This option implies --catch-exceptions.
SimpleLogger Options:
--log-dir=SIMPLE_LOGGER_LOGGER_DIR
Directory for storing log files
--logger-name=SIMPLE_LOGGER_LOGGER_NAME
Name of default logger (and base name of default log file)
--console-log-level=SIMPLE_LOGGER_CONSOLE_LOG_LEVEL
Log to console from this severity up (one of debug, info, warning, error, fatal, none)
--file-log-level=SIMPLE_LOGGER_FILE_LOG_LEVEL
Log to file from this severity up (one of debug, info, warning, error, fatal, none)
--stdout-log-level=SIMPLE_LOGGER_STDOUT_LOG_LEVEL
Log to stdout up to and including this severity.
Higher severities will go to stderr(one of debug, info, warning, error, fatal, none)
--console-log-format=SIMPLE_LOGGER_CONSOLE_LOG_FORMAT
Log to console using selected format (one of short, long)
--file-log-format=SIMPLE_LOGGER_FILE_LOG_FORMAT
Log to file using selected format (one of short, long)
--custom-log-format=SIMPLE_LOGGER_CUSTOM_LOG_FORMAT
Custom log format to be used when --<console/file>-log-format=custom
--log-rotation-files=ROTATION_FILES
Number of log-rotated log files to store
--log-rotation-max-size=ROTATION_MAX_SIZE
Size (in bytes) of log before being log-rotated
--log-rotation-error-files=ROTATION_ERROR_FILES
Number of log-rotated error-log files to store
--log-rotation-error-max-size=ROTATION_ERROR_MAX_SIZE
Size (in bytes) of error-log before being log-rotated
10. lastline_prune_analysis_backlog
Usage: lastline_prune_analysis_backlog [-h] --age-minutes AGE_MINUTES
[--limit LIMIT] [--backend BACKEND]
[--verbose]
optional arguments:
-h, --help Show this help message and exit
--age-minutes AGE_MINUTES
Prune tasks older than this many minutes
--limit LIMIT Prune at max this many tasks
--backend BACKEND Prune tasks for only this analysis backend (optional)
--verbose Enable verbose logging
11. lastline_prune_docker_images
Usage: Script to find old docker images and remove them (from the local system)
NOTE: This command is intended for advanced users only. As the docker documentation states, it's
not entirely trivial to understand what `docker image purge` does. To help with this, this tool
lists images that may be interesting to delete and allows the user to delete them... but, it's
not a fits-all tool... by far...
https://docs.docker.com/engine/reference/commandline/image_prune/
:Copyright:
Copyright 2018 Lastline, Inc. All Rights Reserved.
Optional arguments:
-h, --help Show this help message and exit
--config-file CONFIG_FILE, -c CONFIG_FILE
Specify configuration file.
--keep-days KEEP_DAYS. Keep images that are younger than this (in days)
--keep-tags [KEEP_TAGS [KEEP_TAGS ...]]
Keep images whose tags match this regular expression.
Allows setting multiple, space-separated values
--keep-repositories [KEEP_REPOSITORIES [KEEP_REPOSITORIES ...]]
Keep all images of this repository. Allows setting multiple, space-separated values
--keep-images [KEEP_IMAGES [KEEP_IMAGES ...]]
Keep these images (name:tag). Allows setting multiple, space-separated values
--delete-images [DELETE_IMAGES [DELETE_IMAGES ...]]
Delete these images (regexp on the full name:tag), even if they match an expression provided via one
of the other --keep-* parameters. Allows setting multiple, space-separated values
--system-prune Trigger `docker system prune` after removing images to remove intermediary dangling images this
script may not be able to find
--force Force deletion of images (see `docker image remove --force`)
--assume-yes Delete images without confirmation
SimpleLogger Options:
--log-dir SIMPLE_LOGGER_LOGGER_DIR
Directory for storing log files
--logger-name SIMPLE_LOGGER_LOGGER_NAME
Name of default logger (and base name of default log file)
--console-log-level {none,debug,info,warning,error,critical}
Log to console from this severity up (one of debug, info, warning, error, fatal, none)
--file-log-level {none,debug,info,warning,error,critical}
Log to file from this severity up (one of debug, info, warning, error, fatal, none)
--stdout-log-level {none,debug,info,warning,error,critical}
Log to stdout up to and including this severity. Higher severities will go to stderr(one of debug, info,
warning, error, fatal, none)
--console-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to console using selected format (one of short, long)
--file-log-format {long_process_thread,long_thread,short,json,long,custom}
Log to file using selected format (one of short, long)
--custom-log-format SIMPLE_LOGGER_CUSTOM_LOG_FORMAT
Custom log format to be used when --<console/file>-log-format=custom
--log-rotation-files ROTATION_FILES
Number of log-rotated log files to store
--log-rotation-max-size ROTATION_MAX_SIZE
Size (in bytes) of log before being log-rotated
--log-rotation-error-files ROTATION_ERROR_FILES
Number of log-rotated error-log files to store
--log-rotation-error-max-size ROTATION_ERROR_MAX_SIZE
Size (in bytes) of error-log before being log-rotated
12. lastline_worker_registration
This tool has been deprecated. Please use "lastline_register" instead.
To re-register the Engine appliance after a change of hardware, use
lastline_register --force-register-all-workers
13. get_tres_bundle.py
Usage:
This script uses the Lastline Analyst API to gather diagnostic information about a task.
Run the script with option --help for usage information.
:Copyright:
Copyright 2019 Lastline, Inc. All Rights Reserved.
positional arguments:
task_uuid Identifier (UUID) for the Task
optional arguments:
-h, --help Show this help message and exit
-c CONFIG_FILE, --config-file CONFIG_FILE
Read configuration (such as credentials/analysis url) from this file
-o OUTPUT_FILE, --output-file OUTPUT_FILE
Save the TRES Bundle zip file in this location. By default, the file is saved in the current directory with a
named based on the task UUID.
-s, --include-analysis-subject Include the submitted sample artifact in the bundle.
14. get_backup_status.py
Usage:
Get Backup Summary Status
The full status of the recent backups is dumped in JSON format to standard output.
EXAMPLES:
get_backup_status.py 7
Retrieves a backup status summary from the past week for the appliance
in which the script is ran on.
get_backup_status.py 14 --username USERNAME --password PASSWORD
Retrieves a backup status summary from the past 14 days using the
USERNAME and PASSWORD provided or authentication.
WARNING: When using this approach, please be aware that the
password will be visible in the process list. It is highly recommended
to store credentials in a configuration fi1le, that is not readable by all, instead.
get_backup_status.py --appliance-uuid UUID1 --verbose --config papi_config.ini 20
Retrieves a backup status summary for the past 20 days for the appliance UUID1
with verbose information using credentials from the "papi_config.ini" file.
DUMP ATTRIBUTES
Section contains information about the attributes in the dump.
* summary: The summary of the recent backup status.
* status: The status of the recent backup in the queried time interval:
- "OK": The most recent backups were completed with no errors
- "Warning": There were successful backups; however, errors were present
after the last successful backup
- "Error": There were no successful backups in the specified interval
* successful_backups: A list of metadata of successful backups where each element in the list
is an object of a backup type. Each object within the list contains the following fields:
- "type": The type of backup, e.g: "inc" for Incremental, "full" for full
- "number_of_backups": The number of backups for the backup type
- "most_recent_time": The most recent backup (completion) time for the type
* errors_after_most_recent_backup: A list of all the errors that occurred AFTER the most recent
successful backup (or all the errors if no backups occured). Each object in the list
contains the following attributes:
- "impact_level": The impact level of the error, will either be "Error" or "Warning".
- "start_time": The reported start time of the error
- "end_time": The reported end time of the error
- "component": The component of the error
- "type": The component type that triggered the error
- "message": The reported error message of the error
* query_start_date: The start date of the query used to lookup backup information
* query_end_date: The end date of the query used to lookup backup information.
EXAMPLE RESULT:
{
"status": "Warning",
"successful_backups": [
{
"number_of_backups": 1,
"type": "full",
"most_recent_time": "2017-01-30 18:14:38"
},
{
"number_of_backups": 1,
"type": "inc",
"most_recent_time": "2017-01-31 18:11:36"
}
],
"errors_after_most_recent_backup": [
{
"impact_level": "Error",
"start_time": "2017-01-31 18:20:40",
"component": "Backup Service",
"end_time": "2017-01-31 18:20:40",
"message": "Inc backup "TEST1" failed: A network error occurred",
"type": "Completed Incremental Backups"
},
{
"impact_level": "Error",
"start_time": "2017-01-31 18:25:40",
"component": "Backup Service",
"end_time": "2017-01-31 18:25:40",
"message": "Inc backup "TEST1" failed: A network error occurred",
"type": "Completed Incremental Backups"
}
],
"query_start_date": "2017-01-30",
"query_end_date": "2017-02-01",
"summary": "Successful backups were completed; however, there were additional errors
and/or warnings reported"
}
Positional arguments:
days Number of previous days, including today, from which to look up backup status logs
Optional arguments:
-h, --help Show this help message and exit
--appliance-uuid APPLIANCE_UUID
Unique identifier of an appliance on which we want information
-v, --verbose Displays verbose information about recent backups
Configuration arguments:
-c CONFIG, --config CONFIG Configuration file name
--section SECTION Section of configuration file to read from, defaults to 'papi'
Authentication arguments:
-u USERNAME, --username USERNAME
Use this username instead of the username provided in the configuration file
-p PASSWORD, --password PASSWORD
Use this password instead of the password provided in the configuration file. WARNING: It is not
recommended to use this option as it will show the password in the process list
Additional Information
Feedback
