azure probe configuration will not save

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

I have installed the azure probe and provided the connection information to azure.  I am not sure that this is working correctly and i get some error messages in the log about connecting.

Environment

Release : 20.1

Component : UIM - AZURE

Note that this same issue may occur with other probes when accessing the Admin Console via HTTPS, e.g., netapp_ontap.

Cause

- when performing probe configuration via the Admin Console via HTTPS (payload truncation issue caused by current embedded tomcat version)

Resolution

wasp-20.1.0-hf1 which contains a new tomcat version/jar files that need to be copied into place as per the included directions in the zip release notes.

Note that the wasp package is contained within the zip so you have to extract it first. The release notes are in the zip as well but here they are...

=====================

Hotfix: wasp-20.1.0-hf1
Release Date: 3 June 2020
Applicable for: UIM 20.1.0

======================

This hotfix addresses the following:

Azure probe configuration will not save when accessing and using the Admin Console via HTTPS.

Steps to apply the fix:

1. Deactivate wasp probe in UIM Server Robot.

2. In UIM Server installation location, delete 'work' folder from the path \probes\service\wasp

3. Take a backup of of the following jars from the path \probes\service\wasp\lib and delete those jars from the directory.

tomcat-annotations-api
tomcat-api
tomcat-catalina
tomcat-coyote
tomcat-dbcp
tomcat-el-api
tomcat-jasper
tomcat-jasper-el
tomcat-jaspic-api
tomcat-jdbc
tomcat-jni
tomcat-jsp-api
tomcat-juli
tomcat-servlet-api
tomcat-util
tomcat-util-scan

4. Unzip wasp-20.1.0-hf1.zip and copy all of the jar files to the location \probes\service\wasp\lib

6. Activate the wasp probe.

Additional Information

Here are some possible symptoms/errors based on some similar cases:

Issue occurs when using HTTPS to access the Admin Console and trying to configure probes, and/or trying to save configuration or template changes. Configuration will not save, and/or change to a template will not complete.

Examples of F12 browser debug output when accessing Admin Console via HTTPS and trying to save the configuration or a change to a template are presented below:

'Failed handling failure response. Unable to get property 'allowResubmit' of undefined or null reference'

‘Response was not in JSON format, was: An internal server error has occurred. main.js:453’

‘Response was not in JSON format, was: An internal server error has occurred. main.js:447’

‘Failed handling failure response. Cannot read main.js:453
property 'allowResubmit' of undefined’

In the ppm.log on the admin console machine, you may see: 

‘main.js:453 Response was not in JSON format, was: An internal server error has occurred.’

…

‘main.js:447 Response was not in JSON format, was: An internal server error has occurred.’

…
‘main.js:453 Failed handling failure response. Cannot read property 'allowResubmit' of undefined
p @ main.js:453’

…

Analysis procedures:

Check the mps.log after setting the mps properties file setting into DEBUG mode and reproducing the issue by trying to save the config or make a change to a template.

As we found with the azure probe, its possible based on the error in the browser developer tool (F12), a similar problem occurs when accessing the netapp_ontap probe via Admin Console when using HTTPS:

'Response was not in JSON format, was: An internal server error has occurred.
Failed handling failure response. Unable to get property 'allowResubmit' of undefined or null reference'

The JSON payload is being truncated due to the use of HTTPS when accessing/using the Admin Console, by some tomcat limit/issue, and if this is the same/similar issue, it should be traceable/revealed by examining the mps.log when you try and Save the configuration OR a change to a template.

So to better analyze/reveal the problem please do the following:

1. Deactivate wasp
2. Increase the logging level for mps:

     Edit mps.log4j.properties under ...\Program Files (x86)\Nimsoft\probes\service\wasp\conf

Change WARN to DEBUG for the section below:

# Set loggers for third_party components to WARN level.
log4j.logger.org.springframework=DEBUG
log4j.logger.org.apache.cxf=DEBUG
log4j.logger.org.apache.http=DEBUG

3. Clear out the contents of the mps.log file and save it.
4. Activate the wasp on the Primary
5. Then try to save the netapp_ontap configuration or template again.

The mps.log which should now contain the DEBUG messages.

It should reveal an error with a 'Caused by:' entry like this:

Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input in field name

or

com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input within/between Object entries
 at [Source: (ByteArrayInputStream); line: 1, column: 180091] (through reference chain: com.nimsoft.mons.model.ctd.display.impl.CtdDisplayRootImpl["children"]->java.util.ArrayList[0]->com.nimsoft.mons.model.ctd.display.impl.CtdDisplayEntityImpl["children"]->java.util.ArrayList[0]->com.nimsoft.mons.model.ctd.display.impl.CtdDisplayEntityImpl["children"]->java.util.ArrayList[16]->com.nimsoft.mons.model.ctd.display.impl.CtdDisplayEntityImpl["monitors"]->java.util.ArrayList[0])

Result: cannot save a configuration or a change to a probe template. Probe’s JSON payload gets truncated by Tomcat.

Note that smaller payloads may not end up causing the same truncation issue.

=====================================================================================================================

Broadcom development fixed this issue by upgrading the embedded Tomcat version from 9.0.31 to 9.0.35 due to an issue in the 9.0.31 version of Tomcat.

=====================================================================================================================

wasp zip (wasp-20.1.0-hf1.zip) contains the new bundled version of Tomcat - it is attached to this KB article. It contains both the patch and release notes.

Note that the new jar files are contained within the zip so you have to extract it first. The release notes are in the zip as well but here they are listed below.

Hotfix: wasp-20.1.0-hf1
Release Date: 3 June 2020
Applicable for: UIM 20.1.0

This hotfix addresses the following:

Azure probe configuration will not save in AdminConsole when accessing AC using HTTPS. (Note that as stated, this may affect other probes)

Steps to apply the fix:

1. Deactivate wasp probe in UIM Server Robot.

2. In UIM Server installation location, delete 'work' folder from the path <UIM_Installation>\probes\service\wasp
  
3. Take back up of of the following jars from the path <UIM_Installation>\probes\service\wasp\lib and delete those jars from the directory.

 tomcat-annotations-api
 tomcat-api
 tomcat-catalina
 tomcat-coyote
 tomcat-dbcp
 tomcat-el-api
 tomcat-jasper
 tomcat-jasper-el
 tomcat-jaspic-api
 tomcat-jdbc
 tomcat-jni
 tomcat-jsp-api
 tomcat-juli
 tomcat-servlet-api
 tomcat-util
 tomcat-util-scan

4. Unzip wasp-20.1.0-hf1.zip and copy all jar files to the location <UIM_Installation>\probes\service\wasp\lib

6. Activate wasp probe.

To be clear, essentially this is an update of jar files not a deployment of a new wasp probe per se.

If you access the Admin Console via HTTPS, and you not only use the Admin Console off of the Primary Hub, but also a remote hub, you should follow the steps and deploy the updated jars there as well.

Results: The JSON payload should no longer be truncated and you should be able to save the probe configuration and/or save a change to a template.