MSMTC went down with RACF error because the dataset does not have a user alias defined, so it fails when it is trying to be allocated : IKJ56893I DATA SET MSMUSR.JVM.MSMTC.D200427.T190000.X001 NOT ALLOCATED+
JAVA abended and wanted to write a dump and could not due to permission missing. After defining an alias for msmuser so the MSMUSR.JVM.xxxx dataset / dump could be created and MSMTC failed with these messages:
08.03.35 STC85025 JVMDUMP039I Processing dump event "gpf", detail "" at 2020/04/28 371 371 08:03:35 - please wait. 371 08.03.35 STC85025 JVMDUMP032I JVM requested System dump using 'MSMUSR.JVM.MSMTC.D200428. 372 372 T080335.X&DS' in response to an event 372 08.03.35 STC85025 IGD101I SMS ALLOCATED TO DDNAME (SYS00009) 373 373 DSN (MSMUSR.JVM.MSMTC.D200428.T080335.X001 ) 373 STORCLAS (SCNORM) MGMTCLAS (DEFAULT) DATACLAS (DCMVOL) 373 VOL SER NOS= TLRG07 08.03.41 STC85025 IEA827I COMPLETE TRANSACTION DUMP WRITTEN TO 00001 DATASETS 374 374 STARTING FROM 'MSMUSR.JVM.MSMTC.D200428.T080335.X001' 374 PLEASE COMBINE USING COPYDUMP PRIOR TO VIEWING IN IPCS 08.03.41 STC85025 IGD104I MSMUSR.JVM.MSMTC.D200428.T080335.X001 RETAINED, DDNAME=SYS00009 08.03.41 STC85025 JVMDUMP032I JVM requested Java dump using '/u/camsm/msmtmp/javacore.20 376 376 200428.080335.67109329.0002.txt' in response to an event 376 08.03.41 STC85025 JVMDUMP032I JVM requested Snap dump using '/u/camsm/msmtmp/Snap.202004 377 377 28.080335.67109329.0003.trc' in response to an event 377 08.03.41 STC85025 IEA995I SYMPTOM DUMP OUTPUT 378 378 SYSTEM COMPLETION CODE=0C4 REASON CODE=00000010 378 TIME=08.03.41 SEQ=01434 CPU=4002 ASID=003E 378 PSW AT TIME OF ERROR 478D0001 A06429D0 ILC 4 INTC 10 378 NO ACTIVE MODULE FOUND 378 NAME=UNKNOWN 378 DATA AT PSW IS UNAVAILABLE AT THIS TIME 378 GR 0: 00000010 1: 204296D8 378 2: 20DAC710 3: 00000000 378 4: 00000000 5: 00000000 378 6: 00000000 7: 00000000 378 8: 00000000 9: 00000000 378 A: 00000000 B: 00000000 378 C: 00000000 D: 20429F70 378 E: 00FD4740 F: 206429D1 378 END OF SYMPTOM DUMP 08.03.41 STC85025 IEF450I MSMTC MSMTC - ABEND=S0C4 U0000 REASON=00000010 379 379 TIME=08.03.41 08.03.41 STC85025 IEF404I MSMTC - ENDED - TIME=08.03.41 08.03.41 STC85025 $HASP395 MSMTC ENDED - ABEND=S0C4
Unhandled exception Type=Segmentation error vmState=0x00000000 J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000035 Handler1=000000500871EFB0 Handler2=000000500863F720 gpr0=000000500010AB10 gpr1=00000050082FB300 gpr2=00000050082FB418 gpr3=0000005012136DD0 gpr4=00000050082FB50C gpr5=000000000000003E gpr6=0000005012136E0A gpr7=0000005000F87D00 gpr8=000000001D54B000 gpr9=000000001F50B6D8 gpr10=0000000000000000 gpr11=0000005000100000 gpr12=0000005000108398 gpr13=00000050082FB388 gpr14=0000000020A66F2E gpr15=000000001F50B5A2 fpc=00080000 psw0=0785040180000000 psw1=000000001F50B6A2 sp=00000050082FAA80 bea=000000001F50B676 fpr0=403e000000000000 fpr1=406f000000000000 fpr2=0000000000000000 fpr3=0000000000000000 fpr4=c03dffffffffe800 fpr5=4182666666666667 fpr6=0000000000000000 fpr7=0000000000000000 fpr8=0000000000000000 fpr9=0000000000000000 fpr10=0000000000000000 fpr11=0000000000000000 fpr12=0000000000000000 fpr13=0000000000000000 fpr14=0000000000000000 fpr15=0000000000000000 Target=2_90_20200108_436782 (z/OS 02.04.00) CPU=s390x (3 logical CPUs) (0x2a8d89000 RAM) ----------- Stack Backtrace ----------- protectedIntrospectBacktraceSymbols+0xdf146ea0 (0x0000000000000000) omrsig_protect+0xdf149b88 (0x0000000000000000) omrintrospect_backtrace_symbols+0xdf146f90 (0x0000000000000000) generateDiagnosticFiles+0x835cb840 (0x0000000000000000) omrsig_protect+0xdf149b88 (0x0000000000000000) structuredSignalHandlerVM+0x835ca0f0 (0x0000000000000000) masterSynchSignalHandler+0xdf14d5e8 (0x0000000000000000) __zerro+0xdf4ec1e0 (0x0000000000000000) __zerros+0xdf4ec510 (0x0000000000000000) CEEHDSP+0xdf9d4bb0 (0x0000000000000000) CEEOSIGJ+0xdf727de0 (0x0000000000000000) CELQHROD+0xdf9c1278 (0x0000000000000000) CEEOSIGG+0xdf72ee00 (0x0000000000000000) CELQHROD+0xdf9c1278 (0x0000000000000000) open+0xdf599388 (0x0000000000000000) atoe_open+0xdf206118 (0x0000000000000000) omrfile_open+0xdf1725b0 (0x0000000000000000) zip_openZipFile+0x833de170 (0x0000000000000000) vmizip_openZipFile+0x8355be30 (0x0000000000000000) initializeClassPathEntry+0x8359c620 (0x0000000000000000) findLocallyDefinedClass+0x8344c318 (0x0000000000000000) loadNonArrayClass+0x835e7a80 (0x0000000000000000) internalFindKnownClass+0x835e40d8 (0x0000000000000000) initializeRequiredClasses+0x85ae0ea0 (0x0000000000000000) standardInit+0x85a9d7d0 (0x0000000000000000) J9VMDllMain+0x85a836f8 (0x0000000000000000) runJ9VMDllMain+0x835a2718 (0x0000000000000000) pool_do+0x833c4a40 (0x0000000000000000) runInitializationStage+0x8359eb48 (0x0000000000000000) protectedInitializeJavaVM+0x835a3d40 (0x0000000000000000) omrsig_protect+0xdf149b88 (0x0000000000000000) initializeJavaVM+0x83597138 (0x0000000000000000) J9_CreateJavaVM+0x835b25d8 (0x0000000000000000) JNI_CreateJavaVM+0x832eed58 (0x0000000000000000) JNI_CreateJavaVM+0xdf213280 (0x0000000000000000) JzosVM::createJvm()+0xdfbf9e90 (0x0000000000000000) JzosVM::run(int,char**)+0xdfbfcad8 (0x0000000000000000) main+0xdfbfd2c8 (0x0000000000000000) --------------------------------------- JVMDUMP039I Processing dump event "gpf", detail "" at 2020/04/28 08:03:35 - please wait. JVMDUMP032I JVM requested System dump using 'MSMUSR.JVM.MSMTC.D200428.T080335.X&DS' in response to an event IEATDUMP in progress with options SDATA=(LPA,GRSQ,LSQA,NUC,PSA,RGN,SQA,SUM,SWA,TRT) IEATDUMP success for DSN='MSMUSR.JVM.MSMTC.D200428.T080335.X&DS' JVMDUMP040I System dump written to dataset(s) using name template MSMUSR.JVM.MSMTC.D200428.T080335.X&DS JVMDUMP032I JVM requested Java dump using '/u/camsm/msmtmp/javacore.20200428.080335.67109329.0002.txt' in response to an event JVMDUMP010I Java dump written to /u/camsm/msmtmp/javacore.20200428.080335.67109329.0002.txt JVMDUMP032I JVM requested Snap dump using '/u/camsm/msmtmp/Snap.20200428.080335.67109329.0003.trc' in response to an event JVMDUMP010I Snap dump written to /u/camsm/msmtmp/Snap.20200428.080335.67109329.0003.trc JVMDUMP007I JVM Requesting JIT dump using '/u/camsm/msmtmp/jitdump.20200428.080335.67109329.0004.dmp' JVMDUMP010I JIT dump written to /u/camsm/msmtmp/jitdump.20200428.080335.67109329.0004.dmp JVMDUMP013I Processed dump event "gpf", detail "". JVMJZBL1044E The Java virtual machine aborted
Resolution
The IBM Tivoli Asset Discovery (TAD/z) --- USS(x) parameter for its monitoring task - HSIJMON was set to Y the day before. This caused the JAVA errors, once set back to N, MSMTC came up fine.