AMQ119014: Timed out after waiting 600,000 ms for response when sending packet
Article ID: 245333
Updated On:
Products
Issue/Introduction
After updating to 14.3 or 14.4 and moving to Jboss 7.2 or Wildfly 15 you may occasionally see IDM become un-responsible with the following in the server.log:
AMQ119014: Timed out after waiting 600,000 ms for response when sending packet 71
AMQ119014: Timed out after waiting 600,000 ms for response when sending packet 49
Environment
Component : IdentityMinder (Identity Manager both stand alone and Virtual)
Release : 14.3 14.4 Prior to CP2
The default ttl configuration have been updated in the CP2 release to the values described in this KB.
Some clients might need to adjust further, but most of the client base should not encounter this issue once at 14.4 Cp2.
Cause
Default timeout values are not long enough.
Resolution
For non-VAPP Deployments:
Edit the standalone being used, for example /jboss/standalone/configuration/ca-standalone-full-ha.xml, and change connection-ttl="600000" and call-timeout="600000" for the pooled-connection-factor, and both cluster-connection address="jms"
<pooled-connection-factory call-timeout="600000" client-failure-check-period="3000" confirmation-window-size="1048576" connection-ttl="600000" connectors="in-vm" entries="java:/JmsXA" ha="true" name="activemq-ra" reconnect-attempts="-1" transaction="xa"> <outbound-config allow-local-transactions="true"/> </pooled-connection-factory>
<cluster-connection address="jms" call-timeout="600000" check-period="1000" confirmation-window-size="32000" connection-ttl="600000" connector-name="http-connector" discovery-group="dg-group1prod" message-load-balancing-type="ON_DEMAND" name="my-cluster"/>
<cluster-connection address="jms" call-timeout="600000" check-period="1000" confirmation-window-size="32000" connection-ttl="600000" connector-name="http-connector" discovery-group="dg-group1prod" message-load-balancing-type="ON_DEMAND" name="my-cluster"/>
Where my-cluster = your defined cluster name, should not need to be altered.
Restart the Wildfly/JBoss.
VAPP Information:
We do not have direct access to update the standalone files in VAPP and will need to use CLI commands to make the following updates.
Directions to run these CLI commands:
Add a new user to JBoss:
sudo /opt/CA/wildfly-idm/bin/add-user.sh
What type of user do you wish to add?
a) Management User (mgmt-users.properties)
b) Application User (application-users.properties)
(a): a
Username : YourJbossAdmin
What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[ ]:
Press Enter to leave blank
About to add user 'YourJbossAdmin' for realm 'ManagementRealm'
Is this correct yes/no? yes
Added user 'YourJbossAdmin' to file '/opt/CA/wildfly-idm/standalone/configuration/mgmt-users.properties'
Added user 'YourJbossAdmin' to file '/opt/CA/wildfly-idm/domain/configuration/mgmt-users.properties'
Added user 'YourJbossAdmin' with groups to file '/opt/CA/wildfly-idm/standalone/configuration/mgmt-groups.properties'
Added user 'YourJbossAdmin' with groups to file '/opt/CA/wildfly-idm/domain/configuration/mgmt-groups.properties'
Is this new user going to be used for one AS process to connect to another AS process?
e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
yes/no? no
Then, Navigate to /opt/CA/wildfly-idm/bin/
execute ./jboss-cli.sh
type: connect
Enter credentials
Then run the CLI commands:
For cluster-connection element:
/subsystem=messaging-activemq/server=default/cluster-connection={my-cluster}:write-attribute(name=connection-ttl,value=600000)
/subsystem=messaging-activemq/server=default/cluster-connection={my-cluster}:write-attribute(name=call-timeout,value=600000)
Where my-cluster = your defined cluster name, you can review the standalone file to verify what the cluster name should be, this must be entered correctly in the CLI command shown above
For pooled-connection-factory element:
/subsystem=messaging-activemq/server=default/pooled-connection-factory=activemq-ra:write-attribute(name=connection-ttl,value=600000)
/subsystem=messaging-activemq/server=default/pooled-connection-factory=activemq-ra:write-attribute(name=call-timeout,value=600000)
Restart the Wildfly
Feedback
