After recently migrating from one SQL version to another we are seeing various errors in the nolio_dm_all.log files. In addition to seeing these errors we found that deployments would sometimes hang for a long time. We have tried restarting Nolio services on the managemnet server and and clearing the hibernate cache to resolve the issue. Yet the messages and behaviors persist.
Messages include:
Release : 6.6
Component : CA RELEASE AUTOMATION RELEASE OPERATIONS CENTER
This can happen due to the maxTotal db connections being reached. It can also be related to other environmental related jdbc/mssql connection problems.
To check if the number of active connections is close to, or at, the current maxTotal value, please access the JMX -> noliocenter:type=DataSource. Then, in the Attributes section of this page, take note of your maxTotal and NumActive values.
JMX on the NAC, by default, is accessible via: http://mgmtservername:20203
After making the following changes the errors outlined in the Issue/Introduction section did not reoccur:
Important:
The following update can also be considered if you're using MSSQL and using an AlwaysOn Availability Group. If you're not, or if you're not sure, then do not make this modification.
The "multiSubnetFailure=true" is a setting that can/should only be attempted if:
For more information about the MSSQL AlwaysOn, Availability Groups and the multiSubnetFailover option please consult appropriate Microsoft documentation. The following URLs may offer a starting point:
Note:
The changes above did not resolve the hung deployment problems. This behavior is often associated with MSSQL server resource utilization exhaustion (ex: Memory and/or CPU). Please consult with your DBA to investigate the performance/utilization of the MSSQL Server.
Regarding database connections, you can see how many connections the management server is making to the database using the management server's jmx console. To do this:
If tracing is necessary then please review the following KB article: NAC & SQLException Tracing