search cancel

MSMTC went down with JAVA dump - S0C4-10

book

Article ID: 189728

calendar_today

Updated On:

Products

Mainframe Software Manager (Chorus Software Manager) CHORUS SOFTWARE MANAGER

Issue/Introduction

 MSMTC went down with RACF error because the dataset does not have a user alias defined, so it fails when it is trying to be allocated :
IKJ56893I DATA SET MSMUSR.JVM.MSMTC.D200427.T190000.X001 NOT ALLOCATED+       



19:00:00 27APR MSMTC    002C S84067   19.00.00 ICH408I USER(msmuser  ) GROUP(STCGRP  ) NAME(MSM                 )               
19:00:00 27APR MSMTC    002C S84067   19.00.00   CATALOG.MSTTC40 CL(DATASET ) VOL(volser)                                      
19:00:00 27APR MSMTC    002C S84067   19.00.00   INSUFFICIENT ACCESS AUTHORITY                                                 
19:00:00 27APR MSMTC    002C S84067   19.00.00   FROM CATALOG.M* (G)                                                           
19:00:00 27APR MSMTC    002C S84067   19.00.00   ACCESS INTENT(UPDATE )  ACCESS ALLOWED(READ   )                               
19:00:00 27APR MSMTC    002C S84067   19.00.00 IKJ56893I DATA SET MSMUSR.JVM.MSMTC.D200427.T190000.X001 NOT ALLOCATED+         
19:00:00 27APR MSMTC    002C S84067   19.00.00 IGD17102I CATALOG ERROR IN DEFINING NONVSAM DATA SET                            
19:00:00 27APR MSMTC    002C S84067   19.00.00 MSMUSR.JVM.MSMTC.D200427.T190000.X001                                           
19:00:00 27APR MSMTC    002C S84067   19.00.00 RETURN CODE IS 56 REASON CODE IS 6 IGG0CLFT                                     
19:00:00 27APR MSMTC    002C S84067   19.00.00 IGD306I UNEXPECTED ERROR DURING IGG0CLFT PROCESSING                             
19:00:00 27APR MSMTC    002C S84067   19.00.00 RETURN CODE 56 REASON CODE 6                                                    
19:00:00 27APR MSMTC    002C S84067   19.00.00 THE MODULE THAT DETECTED THE ERROR IS IGDVTSCU                                  
19:00:00 27APR MSMTC    002C S84067   19.00.00 SMS MODULE TRACE BACK - VTSCU VTSCT VTSCR SSIRT

Environment

Release : 6.0

Component : Chorus Software Manager

Cause

JAVA abended and wanted to write a dump and could not due to permission missing.
After defining an alias for msmuser so the MSMUSR.JVM.xxxx dataset / dump could be created and MSMTC failed with these messages:

08.03.35 STC85025  JVMDUMP039I Processing dump event "gpf", detail "" at 2020/04/28  371
   371             08:03:35 - please wait.
   371
08.03.35 STC85025  JVMDUMP032I JVM requested System dump using 'MSMUSR.JVM.MSMTC.D200428.  372
   372             T080335.X&DS' in response to an event
   372
08.03.35 STC85025  IGD101I SMS ALLOCATED TO DDNAME (SYS00009)  373
   373                     DSN (MSMUSR.JVM.MSMTC.D200428.T080335.X001       )
   373                     STORCLAS (SCNORM) MGMTCLAS (DEFAULT) DATACLAS (DCMVOL)
   373                     VOL SER NOS= TLRG07
08.03.41 STC85025  IEA827I COMPLETE TRANSACTION DUMP WRITTEN TO 00001    DATASETS  374
   374             STARTING FROM 'MSMUSR.JVM.MSMTC.D200428.T080335.X001'
   374             PLEASE COMBINE USING COPYDUMP PRIOR TO VIEWING IN IPCS
08.03.41 STC85025  IGD104I MSMUSR.JVM.MSMTC.D200428.T080335.X001        RETAINED,  DDNAME=SYS00009
08.03.41 STC85025  JVMDUMP032I JVM requested Java dump using '/u/camsm/msmtmp/javacore.20  376
   376             200428.080335.67109329.0002.txt' in response to an event
   376
08.03.41 STC85025  JVMDUMP032I JVM requested Snap dump using '/u/camsm/msmtmp/Snap.202004  377
   377             28.080335.67109329.0003.trc' in response to an event
   377
08.03.41 STC85025  IEA995I SYMPTOM DUMP OUTPUT  378
   378             SYSTEM COMPLETION CODE=0C4  REASON CODE=00000010
   378              TIME=08.03.41  SEQ=01434  CPU=4002  ASID=003E
   378              PSW AT TIME OF ERROR  478D0001   A06429D0  ILC 4  INTC 10
   378                NO ACTIVE MODULE FOUND
   378                NAME=UNKNOWN
   378                DATA AT PSW IS UNAVAILABLE AT THIS TIME
   378                GR 0: 00000010   1: 204296D8
   378                   2: 20DAC710   3: 00000000
   378                   4: 00000000   5: 00000000
   378                   6: 00000000   7: 00000000
   378                   8: 00000000   9: 00000000
   378                   A: 00000000   B: 00000000
   378                   C: 00000000   D: 20429F70
   378                   E: 00FD4740   F: 206429D1
   378              END OF SYMPTOM DUMP
08.03.41 STC85025  IEF450I MSMTC MSMTC - ABEND=S0C4 U0000 REASON=00000010  379
   379                     TIME=08.03.41
08.03.41 STC85025  IEF404I MSMTC - ENDED - TIME=08.03.41
08.03.41 STC85025  $HASP395 MSMTC    ENDED - ABEND=S0C4


Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000035
Handler1=000000500871EFB0 Handler2=000000500863F720
gpr0=000000500010AB10 gpr1=00000050082FB300 gpr2=00000050082FB418 gpr3=0000005012136DD0
gpr4=00000050082FB50C gpr5=000000000000003E gpr6=0000005012136E0A gpr7=0000005000F87D00
gpr8=000000001D54B000 gpr9=000000001F50B6D8 gpr10=0000000000000000 gpr11=0000005000100000
gpr12=0000005000108398 gpr13=00000050082FB388 gpr14=0000000020A66F2E gpr15=000000001F50B5A2
fpc=00080000 psw0=0785040180000000 psw1=000000001F50B6A2 sp=00000050082FAA80
bea=000000001F50B676
fpr0=403e000000000000 fpr1=406f000000000000 fpr2=0000000000000000 fpr3=0000000000000000
fpr4=c03dffffffffe800 fpr5=4182666666666667 fpr6=0000000000000000 fpr7=0000000000000000
fpr8=0000000000000000 fpr9=0000000000000000 fpr10=0000000000000000 fpr11=0000000000000000
fpr12=0000000000000000 fpr13=0000000000000000 fpr14=0000000000000000 fpr15=0000000000000000
Target=2_90_20200108_436782 (z/OS 02.04.00)
CPU=s390x (3 logical CPUs) (0x2a8d89000 RAM)
----------- Stack Backtrace -----------
protectedIntrospectBacktraceSymbols+0xdf146ea0 (0x0000000000000000)
omrsig_protect+0xdf149b88 (0x0000000000000000)
omrintrospect_backtrace_symbols+0xdf146f90 (0x0000000000000000)
generateDiagnosticFiles+0x835cb840 (0x0000000000000000)
omrsig_protect+0xdf149b88 (0x0000000000000000)
structuredSignalHandlerVM+0x835ca0f0 (0x0000000000000000)
masterSynchSignalHandler+0xdf14d5e8 (0x0000000000000000)
__zerro+0xdf4ec1e0 (0x0000000000000000)
__zerros+0xdf4ec510 (0x0000000000000000)
CEEHDSP+0xdf9d4bb0 (0x0000000000000000)
CEEOSIGJ+0xdf727de0 (0x0000000000000000)
CELQHROD+0xdf9c1278 (0x0000000000000000)
CEEOSIGG+0xdf72ee00 (0x0000000000000000)
CELQHROD+0xdf9c1278 (0x0000000000000000)
open+0xdf599388 (0x0000000000000000)
atoe_open+0xdf206118 (0x0000000000000000)
omrfile_open+0xdf1725b0 (0x0000000000000000)
zip_openZipFile+0x833de170 (0x0000000000000000)
vmizip_openZipFile+0x8355be30 (0x0000000000000000)
initializeClassPathEntry+0x8359c620 (0x0000000000000000)
findLocallyDefinedClass+0x8344c318 (0x0000000000000000)
loadNonArrayClass+0x835e7a80 (0x0000000000000000)
internalFindKnownClass+0x835e40d8 (0x0000000000000000)
initializeRequiredClasses+0x85ae0ea0 (0x0000000000000000)
standardInit+0x85a9d7d0 (0x0000000000000000)
J9VMDllMain+0x85a836f8 (0x0000000000000000)
runJ9VMDllMain+0x835a2718 (0x0000000000000000)
pool_do+0x833c4a40 (0x0000000000000000)
runInitializationStage+0x8359eb48 (0x0000000000000000)
protectedInitializeJavaVM+0x835a3d40 (0x0000000000000000)
omrsig_protect+0xdf149b88 (0x0000000000000000)
initializeJavaVM+0x83597138 (0x0000000000000000)
J9_CreateJavaVM+0x835b25d8 (0x0000000000000000)
JNI_CreateJavaVM+0x832eed58 (0x0000000000000000)
JNI_CreateJavaVM+0xdf213280 (0x0000000000000000)
JzosVM::createJvm()+0xdfbf9e90 (0x0000000000000000)
JzosVM::run(int,char**)+0xdfbfcad8 (0x0000000000000000)
main+0xdfbfd2c8 (0x0000000000000000)
---------------------------------------
JVMDUMP039I Processing dump event "gpf", detail "" at 2020/04/28 08:03:35 - please wait.
JVMDUMP032I JVM requested System dump using 'MSMUSR.JVM.MSMTC.D200428.T080335.X&DS' in response to an event
IEATDUMP in progress with options SDATA=(LPA,GRSQ,LSQA,NUC,PSA,RGN,SQA,SUM,SWA,TRT)
IEATDUMP success for DSN='MSMUSR.JVM.MSMTC.D200428.T080335.X&DS'
JVMDUMP040I System dump written to dataset(s) using name template MSMUSR.JVM.MSMTC.D200428.T080335.X&DS
JVMDUMP032I JVM requested Java dump using '/u/camsm/msmtmp/javacore.20200428.080335.67109329.0002.txt' in response to an event
JVMDUMP010I Java dump written to /u/camsm/msmtmp/javacore.20200428.080335.67109329.0002.txt
JVMDUMP032I JVM requested Snap dump using '/u/camsm/msmtmp/Snap.20200428.080335.67109329.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /u/camsm/msmtmp/Snap.20200428.080335.67109329.0003.trc
JVMDUMP007I JVM Requesting JIT dump using '/u/camsm/msmtmp/jitdump.20200428.080335.67109329.0004.dmp'
JVMDUMP010I JIT dump written to /u/camsm/msmtmp/jitdump.20200428.080335.67109329.0004.dmp
JVMDUMP013I Processed dump event "gpf", detail "".
JVMJZBL1044E The Java virtual machine aborted

Resolution

The IBM Tivoli Asset Discovery (TAD/z) --- USS(x) parameter for its monitoring task - HSIJMON was set to Y the day before.
This caused the JAVA errors, once set back to N, MSMTC came up fine.