When a component goes down, collect all the log files available from each DevTest component - IAM, Enterprise Dashboard, Registry, Broker, Portal, Coordinator, Simulator, VSE.
Even if just the VSE is going down, it can be that it is a consequence of an issue with a different component.
Verify if there a hprof file was created under the $DEVTEST_HOME/tmp folder. A hprof file can be analyzed with a Memory Analyzer Tool.
Verify if a hs_err_pid file was created under the $DEVTEST_HOME folder. A hs_err_pid file can be opened with a text editor.
These files will provide information that can lead to the root cause of the issue.