DevTest Environmental Restrictions and Communication issues
Article ID: 9872
Updated On:
Products
Issue/Introduction
DevTest is a soft real-time distributed system, and, as such care must be taking when planning an installation or upgrade.
This document aims to assist those wishing to avoid reliability, performance and communication issues when installing and configuring DevTest, and consolidates information found elsewhere within the DevTest documentation.
Environment
All supported DevTest releases.
Cause
N/A
Resolution
The information here should be used in conjunction with the release notes for the DevTest version being configured.
The Database.
DevTest ships with an internal database (Derby) for the Enterprise Dashboard and Registry and H2 for IAM configured that allows for a single machine to be used for evaluation, demonstration or debugging purposes only.
The Derby and H2 databases are not supported for extended use, use in a deployed system or use within a distributed system and must be replaced with an enterprise grade database.
· This enterprise database must be well maintained, and have a high bandwidth/low latency link to DevTest. Database maintenance must be arranged locally.
· It is recommended to ensure that schema level backups are performed as appropriate for your business needs – issues arising from database corruption as a result of database rather than DevTest issues will need to be recovered from a backup or via creation of a new schema.
· DevTest requires three database schemas – one for use by IAM, one use by the Enterprise Dashboard and one by the Registry and other components. These schemas may reside on the same server instance, but must be separate schemas.
· The supported enterprise database types are DB2, Oracle, MySQL, Microsoft SQL Server, PostgreSQL and Azure SQL. For supported versions please consult the release notes for the DevTest release that you are configuring.
· The DevTest product requires that schemas are configured to use UTF-8 character encoding. Should this restriction not be observed then it will not be possible to log in to DevTest.
In-place database schema upgrades are supported – elevated database privileges such as DBA privilege may be required in order to perform such an upgrade.
Please note that, once upgraded, it is not possible to revert to a previous version as the schema will no longer be compatible, is is best to take a backup of all databases before upgrading.
Communication Requirements
Ensure that your firewall allows DevTest Solutions to send and receive network transmissions. The functionality that DevTest Solutions provides requires access to network resources and does not work properly when blocked by a firewall. Authorize DevTest Solutions applications.
All DevTest server components must have access to the External Database the Registry is connecting to. Each DevTest server component communicates directly with the database to record their actions. Any restriction to the flow of this data has adverse effects.
The Network
DevTest relies on good network communication being available at all times, and is dependent upon network performance
· No server component may have a network latency to the database that exceeds 20ms.
· No server component may have a network to the registry or portal that exceeds 20ms.
· The workstation may exceed this 20ms limit, but reliability and performance issues should be expected. If the Workstation round trip time exceeds 100ms then the configuration should be considered unsupported.
Any issues arising from network latency, bandwidth constraints or packet loss will need to be addressed by the local network engineering team or by re-engineering your system.
Name Resolution
DevTest requires that hostname resolution work correctly at all times, in both a forward and reverse direction – that is, resolving a name to an IP address must function correctly, and then resolving that IP address must yield the original host.
NAT (Network Address Translation)
DevTest communicated via ActiveMQ, with messages containing network endpoint information. It is therefore not possible to use DevTest via NAT.
Virtual IP / Network Address Changes.
DevTest does not support the use of Virtual IP or any installation where the network address may change during the lifetime of a server component (Registry, Portal, Broker, Coordinator, Simulator or Virtual Service Environment).
It is necessary for DevTest to resolve a hostname internally to the same address as that perceived from outside the system upon which it is installed, and thus installations using virtual IP will encounter issues
Since the resolution of the hostname is performed at component start a change of address of the running system is also not permitted.
The File System
DevTest writes real-time communication data and log files into a temporary directory. Given the real-time nature of this data, it is required that this temporary directory (commonly referred to as lisatmp) is situated on a local file system. Systems that have lisatmp on a remote file system will experience issues and are not supportable.
This constraint extends to installations that are present within a Virtual Machine (VM) – the disk image used by the VM must reside on local disking. It may be necessary to consult your system administrators in order to determine if this is the case as the disking may appear to be local even if the disk image is remotely hosted.
Connections to the System Under Test (SUT) or Virtual Service Environment
It is necessary to ensure that your Simulators have adequately fast and reliable communications with your SUT, and that the Virtual Service Environments (VSEs) have adequate connectivity to any systems that they depend upon (this includes JMS/MQ environments, live systems, databases used)
Any database connections required for test or virtual service execution must be established via the use of a Type 4 (pure Java) JDBC Connector.
Please consult the database vendor for limitations and configuration.
Integration with 3rd Party Products
DevTest provides interfaces, such as a REST API and JUnit support, that allow integration with third-party products. When configuring these integrations CA Support can supply advice on the use of the interfaces, but cannot provide specific information on a given product.
Where possible it is advisable to exercise the DevTest interface independently of any product in order to determine where any issues lie.
Customizations / Addition Functionality
DevTest may be extended with the addition of customized code that adds such things as test steps and additional protocols to the product.
It is essential that such customizations are compiled against the version of DevTest in use – problems arising from code not compiled against the correct release may be unpredictable and not limited to the additional functionality.
Consult the owner of the customization code to ensure that it has been correctly compiled for use in the release that you have.
Running Mixed Versions
It is not permitted to run mixed versions of the major components of DevTest – that is, the Registry, Portal, Broker, Coordinator, Simulator, Virtual Service Environment and Workstation must all be at the same release.
It is, however, permitted to connect to an Enterprise Dashboard that is of a later version than the other components, for instance
· Connecting a Registry from version 10.6 to an IAM and Enterprise Dashboard to version 10.7.2 is permitted since the Enterprise Dashboard is of a later version than the connecting Registry
· Connecting a 10.7.2 Registry to a 10.6.0 IAM and Enterprise Dashboard is not permitted since the Enterprise Dashboard is of an earlier version than the connecting Registry.
· Starting with DevTest 10.7.0, you may connect a DevTest 10.6.0 Workstation to a DevTest 10.7.2 Registry.
It is therefore recommended that IAM and the Enterprise Dashboard be the first component to be addressed during an upgrade.
Operating System Considerations.
When deploying DevTest it is necessary to consider not only the product but also the environment in which it is running – in addition to ensuring that the system has adequate physical resources there it is also necessary to consider the OS resources that may require adjustment.
Linux Hosts – Maximum Number of Open Files per Process
In particular, on Linux hosts, the maximum number of open files per process may, by default, be configured to a value suitable for interactive use rather than a value suitable for servers.
This value may be configured on a per-user basis, and may be ascertained by entering the command
ulimit -n
at the shell prompt of the user that will be running DevTest.
Should the value returned be less than 10240, then it should be increased. Since the configuration of this value may be vendor specific, please consult the OS vendor’s documentation for instructions on how to increase this value.
Not increasing this value will have adverse effects upon DevTest, with the VSE, Simulator and database machines being most likely to encounter difficulties.
Ephemeral Ports
An ephemeral port is a networking term that refers to the network port used for the outgoing leg of a connection. Each end of a network connection is identified by its network address and its port – typically the destination port is known, and the source port is randomly allocated from the pool of ephemeral ports.
Consider a connection to a DevTest Registry from a Coordinator
The connection is to a known address/port pair, or example tcp://registry.my.net:2010/Registry
where registry.my.net corresponds to a known address, and the port is 2010.
However, this connection must originate from an address/port pair – in this case the coordinator machine will be known in advance, but the port will be randomly selected by the operating system.
If there are not enough ephemeral ports available to support all the connections that are required for every process running on the system, then the ephemeral ports ranges can be extended.
Defaults
The Internet Assigned Numbers Authority (IANA) recommends that the ephemeral port range be 49152 to 65535.
Many Linux Implementations use the range 32768 to 61000.
Microsoft Windows and later IANA range by default.
Apple OS X used the IANA range.
All modern operating systems allow the ranges to be changed, however, and, in the event that this has been done some issues opening sockets may be encountered.
Changing Ephemeral Port Allocations.
As would be expected, changing the ephemeral port is OS specific. A summary of how these may be adjusted on various OS types can be found here http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html. Should this document not provide the appropriate information for your specific operating system and version then it will be necessary to obtain the required instructions from the operating system vendor.
Additional Information
The latest DevTest documentation can be found via https://techdocs.broadcom.com
Ephemeral port allocation change information http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html
Errors like "java.lang.RuntimeException: Cannot load keystore" or "java.io.FileNotFoundException" may indicate a low value for 'ulimit -n'.
Feedback
