Frequently troubles during Mainframe ZOS support
(Don’t trouble trouble until trouble troubles you!)
Contents
2. TSO Logon time delay and system wait
4. Bad SMF records when DUMPXY is restarted after cc=0008
5. Jes2 queue is full – offload on tape and delete
6. WTO/SYSLOG buffers are full and SYSLOG file need to be saved on tape or transfered to PC
7. CICS online application pending in “system” – stop, cancel, restart
7.1. Stop CICS online application
7.2. Restart CICS online application
7.3. VTAM generating messages stop
8. TSO user locked – cancel (forced)
9. Forgotten TSO user password
10. Forgotten CICS/KBI user password
11. Reactivate or add/delete TCPIP routes required for Mainframe
12. Backup/Restore troubles with saved SMF file on tape – DCB parameters
14. The problem “No connection” Edit/View of TSO/ISPF library
1. Duplicated SYSID in 2 LPARs during SMF processing of SCRT Report for Sub-capacity charges of Mainframe
If you have a trouble related to SCRT report not accepted from IBM/LMS because one of the LPAR’s in the CPU of Mainframe has duplicated SYSID with another LPAR on the same CPU, go to article ” Troubles with duplicated SYSID in 2 LPARs”.
2. TSO Logon time delay and system wait
If you or other customers are waiting too much when logging on TSO of z/OS in only one particular LPAR of the CPU and no TSO logon time delay exists on other LPAR’s, go to item 1 of the Contents in article “Troubles with TSO Logon time delay and system wait”.
If TSO is in system wait troubles during normal operations, go to item 2 of the Contents in article “Troubles with TSO Logon time delay and system wait”.
3. SMF summary file is full
When a warning message is received that SMF summary file is full and the job DUMPXY of z/OS (writing regular SMF dumps on SMF summary file) ended with cc=0008 (due to 100% full of SMF summary file) and a new active work SMF file is switched with dump required for the previous active work SMF file, go to article “Troubles when SMF summary file is full” to solve the problem and to avoid this situation in the future.
4. Bad SMF records when DUMPXY is restarted after cc=0008
When the failed job DUMPXY with RC=0008 is resubmitted (as SMF dump was required) and some SMF records are dumped (written ) in a wrong way (bad SMF records) on the summary SMF file resulted in ABEND002 with rc=0004 during the extracting step of SMF records (in preparing the final SCRT Report) , go to article “Troubles with removing bad SMF records” to remove the bad SMF records.
5. Jes2 queue is full – offload on tape and delete
If a message appears that JES2 queue is full and nobody can logon in TSO, then some Emergency actions should be performed to reanimate the system. If a warning message appears that JES2 limit is reached, some actions are necessary to avoid the status of JES2 full. In the all cases above go to item 1 (Emergency actions if warning limit reached or full JES2) of the Contents in article “Troubles when Jes2 queue is full”.
If you have a care about the JES2 queue properly processing and maintenance (not to be 100% full when stops z/OS functioning), go to item 2 (JES2 offload on tape and delete) of the Contents in article “Troubles when Jes2 queue is full” to prevent the possible troubles.
6. WTO/SYSLOG buffers are full and SYSLOG file need to be saved on tape or transfered to PC
When the message area is full and the z/OS system stopped working due to message buffer pool limit reached, go to item 1 of the article ‘Troubles when SYSLOG or WTO buffers are full’ to solve the urgent problem and to learn how to avoid it in the future.
To save the current SYSLOG data on tape a procedure is used for daily/monthly processing of SYSLOG on a production system , minimizing the size of the SYSLOG. Go to item 2 of the article ‘Troubles when SYSLOG or WTO buffers are full’.
To save the current SYSLOG STC**** to PC for later investigating and analysis, z/OS file is created that contains all SYSLOG current data and sent to PC with MF-PC text file transfer. Go to item 3 of the article ‘Troubles when SYSLOG or WTO buffers are full“.
7. CICS online application pending – stop, cancel, restart
If the CICS online application is pending and all CICS users are waiting in “system”, then go to next steps explaining how to stop, cancel and restart the CICS application in correct way.
7.1. Stop CICS online application
First of all the KBI online application should be stopped if possible as follows:
– start a new terminal session from the pending one or from TSO/ISPF session with – File – Run the same – and the following VTAM menu appears appears:
B BC2AC10A Production Kirchman Bankway CICS – enter letter ‘B’ – and sign on in the CICS with ID=xxxxxx PW =yyyyyy.
– start KBI online – enter ‘so01’ (KBI operation transaction name) with KBI ID=zzzzzz PW = pppppp.
– close KBI operation and all KBI files from KBI operation menu.
– close CICS system with the command ‘CEMT P SHUT,I’.
If this is not possible (pending system wait during KBI stop) the command ‘CEMT P SHUT,I’ should be issued from any CICS terminal to try to stop the online system in a regular way.
If the above doesn’t work (KBI and CICS can’t be stopped) then go to TSO/ISPF – sd.st – to cancel the CICS application BC2AC10A with:
‘c BC2AC10A STC03521 BC2AC10A 15 EXECUTION BC2A BC2A ‘
The cancel command could be issued from TSO/ISPF – sd.log – enter ‘/c BC2AC10A’ or from the system main or alternate console with – ‘c BC2AC10A’ to stop forced the CICS application job in z/OS system.
7.2. Restart CICS online application
– restart CICS job in z/OS
To restart the CICS job again, from TSO/ISPF – sd.log – enter ‘/s BC2AC10A’. The same could be done from the system main or alternate console with – ‘s BC2AC10A’.
– restart CICS system in z/OS
When CICS system is restarted from TSO/ISPF – sd.log – start a new terminal session with – File – Run the same – and the following VTAM menu appears appears::
B BC2AC10A Production Kirchman Bankway CICS – enter letter ‘B’ – and sign on in the CICS with ID=xxxxxx PW =yyyyyy.
– restart KBI online application
After CICS is started – enter ‘so01’ (KBI operation transaction name) with KBI ID=zzzzzz PW = pppppp and KBI operation menu appears.
– open KBI operation and all KBI files from KBI operation menu.
– restart KBI online operations
from TSO/ISPF – sd.log – start a new terminal session with – File – Run the same – and the following appears:
B BC2AC10A Production Kirchman Bankway CICS – enter letter ‘B’ – and sign on in the CICS with ID=xxxxxx PW =yyyyyy.
Enter ‘sy01’ (KBI processing transaction name) with KBI ID=zzzzzz PW = pppppp and KBI processing menu appears – KBI is restarted.
7.3. VTAM generating messages stop
If the following VTAM messages appear on syslog (sd.log) – stop them:
IST530I AM GBIND PENDING FROM BC1BVTAM TO BC4AVTAM FOR BC4AVTAM 365
IST1051I EVENT CODE = 0202
IST1062I EVENT ID = 00000002000100000004000102020F3100000005
IST314I END
IST530I AM GBIND PENDING FROM BC1BVTAM TO BC2AVTAM FOR BC2AVTAM 366
IST1051I EVENT CODE = 0202
IST1062I EVENT ID = 00000002000100000001000102020F3100000004
IST314I END
/v net,id=bc4avtam,inact,f – stop communication with BC4AVTAM application
IST097I VARY ACCEPTED
IST617I DEACTIVATION IN PROGRESS FOR BC4AVTAM
IST105I BC4AVTAM NODE NOW INACTIVE
/v net,id=bc1bvtam,inact,f – stop communication with BC1BAVTAM application
IST097I VARY ACCEPTED
IST617I DEACTIVATION IN PROGRESS FOR BC1BVTAM
ST105I BC2AVTAM NODE NOW INACTIVE
8. TSO user locked – cancel (forced)
If TSO user is locked (pending) and can’t sign on (due to not logged off properly with – PF3 – PF3 – PF3 – ‘Logoff’ command or due to some operator or system error), first of all it should be canceled (to unlock) from system operator in TSO/ISPF – sd.log – with:
– /c u=ibmuser – cancel tso ibmuser if pending
– /f u=bca1mgp,arm – cancel tso user forced
– /d ts,a – display active tso users to see if successfuly canceled
After that the successfuly canceled TSO user can logon in TSO session again (with the same VTAM menu session started or from another TSO/ISPF session with – File – Run the same) – and when the following VTAM menu appears:
‘T BC2ATSOA Biochim TSO Production ‘
Enter the letter ‘T’ and follow with the TSO user own ID=zzzzzz and PW = pppppp.
The TSO user can now operate OK with the TSO/ISPF menu.
9. Forgotten TSO user password
To change or reactivate the forgotten TSO user password do the following:
From TSO/ISPF – p.6:
Enter: alu bcp1fts password(passfts) resume
TSO user ID = bcp1fts
TSO user PW = passfts
The new password of TSO UID=bcp1fts is now passfts.
10. Forgotten CICS/KBI user password
10.1. CICS password
To change or reassign the forgotten CICS user password do the following:
From TSO/ISPF – p.6 – issue the command:
alu cu330099 password(cu099) resume – Enter
From VTAM menu session choose CICS/KBI application ‘B’ in LPAR BC2A/90:
B BC2AC10A Production Kirchman Bankway CICS – after ‘B’ entered the following appears:
Type your userid and password, then press ENTER:
Userid . . . . cu330099 Groupid . . .
Password . . . cu099
Language . . .
New Password . . .
Type your userid and password, then press ENTER: – Enter
Userid . . . . CU330099 Groupid . . .
Password . . .
Language . . .
New Password . . .
DFHCE3525 Your password has expired. Please type your new password.
New Password stef123 – Enter
DFHCE3539 Please reenter the new password for verification.
New Password stef123 – Enter
The new password of CICS user CU330099 is stef123 for BC2AC10A/CICS10 in LPAR BC2A/90.
10.2. KBI password
To change or reassign the forgotten KBI password , from VTAM menu session choose CICS/KBI application ‘B’ in LPAR BC2A/90:
B BC2AC10A Production Kirchman Bankway CICS – after ‘B’ entered the following appears:
Type your CICS userid and password, then press ENTER:
Userid . . . . cu330405 Groupid . . . – authorized CICS user ID
Password . . . XXXXXXXX – – authorized CICS user PW
Language . . .
New Password . . .
DFHCE3549 Sign-on is complete (Language ENU).
bs01 – special KBI transaction and security code: XXXXXX entered.
For your KBI USER IDENTIFIER ( cu330099 ) with CURRENT STATUS INACTIVE
enter command bsrest from main BSMENU of bs01 to change STATUS as ACTIVE:
PLEASE CONFIRM YOUR REQUEST ( 1 ) 1 – YES
2 – NO
Enter command bsaspw from BSMENU to ASSIGN the new USER PASSWORD: cuxxx
ASSIGN PASSWORD ( cuxxx )
VERIFY ASSIGNED PASSWORD ( cuxxx ) Enter
Exit the bs01 transaction.
To start KBI application on the same CICS session enter sy01 transaction – the following appears:
PLEASE ENTER YOUR KBI USER-ID
************
* cu330099 * – write KBI ID =cu330099
************
ENTER YOUR PASSWORD
************
* cuxxx * – write KBI new PW=cuxxx – Enter
************
A new PW is required
************
* *
************
To change enter your new PW
************
* stef123 * – Enter
A new PW is required – please enter it now
Repeat your new PW again
************
* stef123 * – Enter
************
The main KBI menu is displayed and you can work with the CICS online application.
11. Reactivate or add/delete TCPIP routes required for Mainframe
If TCPIP configuration is changed it is necessary to display, update and reactivate the TCPIP routes to Mainframe as follows:
– display Routes
From TSO/ISPF – sd.log – enter the following TCPIP command to display active/connected Routes to Mainframe:
/d tcpip,,t,conn – display tcp – 23 routes active,
– Update Routes
To edit the PROFILE.TCPIP, enter the member tcpprof in Library bc2a.zos140.parmlib and find (F) ‘HOME addresses’:
; ======================================================================
; HOME addresses
; ======================================================================
;
; HOME: Provides the list of home IP addresses and associated link names
;
HOME
10.212.4.90 BC2AJES – the current LPAR BC2A IP address
; Routing configuration
; ======================================================================
; ———————————————————————-
; Static routing
; ————————————————–
BEGINROUTES – Here it is necessary to update – add/delete Route!!!
Route 10.212.4.0/24 = BC2AJES MTU 1500
Route 10.150.48.0/16 10.212.4.3 BC2AJES MTU 1500
—————-
ENDROUTES
– activate the new Routes /TSO user should be with AUTHORITY writes – IBMUSER/
From TSO/ISPF – sd.log – enter: /v tcpip,,O,bc2a.zos140.parmlib(tcpprof) – activate tcp after update in bc2a.zos140.parmlib(tcpprof):
12. Backup/Restore troubles with saved SMF file on tape – DCB parameters
If in the restore job (for saved SMF file on tape ) the tape DCB is of not existing file or of existing file (or existing library) without parameters, and the job execution is not successful:
– when the tape DCB in the restore job is of an existing library (DCB=BC1B.ZOS140.OPER.LIB) without params (for saved file SMFDUMPW.D040213 on tape CB0394 with backup using DCB=(IPOSAV.SMFDUMPW), the restore from SMF save tape is not successful – cc=0008 / ADR374E (001)-OPNCL(08), UNABLE TO OPEN DDNAME TAPE, 10/:
//DFDSS EXEC PGM=ADRDSSU,REGION=0M,TIME=1440 PARM=’TYPRUN=NORUN’
//SYSPRINT DD SYSOUT=*
//TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG),
// VOL=SER=CB0394,UNIT=(300,,DEFER),
// DCB=(BC1B.ZOS140.OPER.LIB), or DCB=BC1B.ZOS140.OPER.LIB
// LABEL=(6,SL)
//SYSIN DD *
RESTORE –
DS(INC( –
KCI08.SMFR.D040213 –
)) –
INDDNAME(TAPE) –
RENAMEU(KCI08.SMF.D040213,IPOSAV.D0402R) –
SPHERE
– If the tape DCB in the restore job is not of an existing file (KCI01.DMNSNTAP.MODEL.DCB), the restore from SMF save tape is not successful – JCL ERROR – /IEF212I RESMF02 DFDSS TAPE – DATA SET NOT FOUND/:
//DFDSS EXEC PGM=ADRDSSU,REGION=0M,TIME=1440 PARM=’TYPRUN=NORUN’
//SYSPRINT DD SYSOUT=*
//TAPE DD DSN=BC2A.BKPKCI08.SMF.D180612.D020712,DISP=(NEW,CATLG),
// VOL=SER=CB0344,UNIT=(300,,DEFER),
// DCB =(KCI01.DMNSNTAP.MODEL.DCB –
// RECFM=U,LRECL=0,BLKSIZE=32760,
// DSORG=PS),
The solution is the tape DCB to have the same DCB used during backup (DCB=IPOSAV.SMFDUMPW) – then the restore from tape is successful – cc=0000:
//TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG),
// VOL=SER=CB0394,UNIT=(300,,DEFER),
// DCB=(IPOSAV.SMFDUMPW), – without params
or to have the same DCB=IPOSAV.SMFDUMPW and params as was in backup job:
//TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG),
// VOL=SER=CB0394,UNIT=(300,,DEFER),
// DCB=(IPOSAV.SMFDUMPW,
// RECFM=U,LRECL=0,BLKSIZE=32760,
// DSORG=PS),
or to have the DCB of another existing file (IPOSAV.LOGREC) and params as was in backup job:
//TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG),
// VOL=SER=CB0394,UNIT=(300,,DEFER),
// DCB=(IPOSAV.LOGREC,
// RECFM=U,LRECL=0,BLKSIZE=32760,
// DSORG=PS),
or to have the DCB of existing library (BC1B.ZOS140.OPER.LIB) and the same params as was in backup job:
//TAPE DD DSN=BC1B.SMFDUMPW.D270712.D020812,DISP=(NEW,CATLG),
// VOL=SER=CB0394,UNIT=(300,,DEFER),
// DCB=(BC1B.ZOS140.OPER.LIB,
// RECFM=U,LRECL=0,BLKSIZE=32760,
// DSORG=PS),
The conclusion: for SMF restore tape DCB – to be of a sequental file or to has params with (DSORG=PS) – doesn’t matter how it was backed up – wth DCB of SMF file or with params (DSORG=PS).
– If the tape is from another LPAR do the following to avoid some possible troubles (tape volume not found etc.) :
? Verify if the tape volune l exists (is – r – 5 – 1 – 1 – Display volume information) and if not – add the tape volume to RMM (is – r – 5 – 1 – 2).
? If the tape volume exists but has other params – Delete or release a volume (is – r – 5 – 1 – 4) – add the tape to RMM (is – r – 5 – 1 – 2) with the necessary params.
? If not possible to delete a tape volume label – go to “Case 2. The problem: RMM – error rc=12 when try to release, force any tape” of article “Mainframe Tape utilities” where this problem and the decision are described.
13. No space in a Storage group to restore archive VSAM DB files – add, delete,move VSAM files and disk volumes
If you need a space to restore archive VSAM DB files on existing storage group without adding new disk volumes, the following must be done:
– delete unused VSAM files and garbage as shown in article ZOS VSAM utilities or initialize volumes with the obsolete data
– move/condense VSAM files to free disk space on volumes as follows:
If you have not enough free disk volumes but you have some free space on volumes in a storage group that should be gathered on total free volumes in the same or another storage group (condense VSAM files space), go to “ 1.1. Move(condense) VSAM files to free disk volume/s in a storage group” of “Case1. Copy/move VSAM files and full disk volumes to disk” in article ZOS VSAM utilities. All VSAM files from one or more volumes are copied to volumes in the same storage group with enough available free space and the freed volumes are set offline to avoid using them if second such copy is necessary. The freed volumes must be excluded from the old storage group (disable) and can be set as spare volumes or can be added to another storage group in the same or in another LPAR.
If you need a space to restore archive VSAM DB files on additional new disk volumes, the following must be done:
– add new disk volumes to a storage group as described in article Mainframe_Disk_utilities
– copy a big file on chosen disk volumes with free available space for any storage group
Use the following job COPYB01 to copy file IPOSAV.SMFDUMPW on eligible disk volumes DEV302,DEV281,DEV272,DEV269,DEV262 with free space on them:
//COPYB01 EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD DISP=OLD,DSN=IPOSAV.SMFDUMPW
//SYSUT2 DD DSN=KCI07.SMF.D050313,UNIT=SYSDA,
// VOL=SER=(DEV302,DEV281,DEV272,DEV269,DEV262),
// SPACE=(CYL,(1000,110)),DISP=(NEW,CATLG,DELETE)
Space information for the copied file:
Used cylinders . . : 3,262
Used extents . . . : 22
49650 98 22 3390
14. The problem “No connection” Edit/View of TSO/ISPF library
If you receive a message ‘No Connection’ when you want to edit/view any member of TSO/ISPF library, look at the following explanation.
If you are working in ISPF C/S WSA for file transfers between PC and Mainframe and when finally you exit from this connection with WSDISCON command in ISPF or stop wsa.exe application on PC, you may receive a message in TSO ‘No Connection’ that appears when you want to edit/view any member in edit /view mode of ISPF library. In this case go to article “No connection” Edit problem in TSO/ISPF C/S WSA environment to solve the problem and rid of it.
15. Logrec data set is full
If the Logrec data set is full, look at the following explanation.
If the error recording data set (ERDS also known as LOGREC – logrec data set) is full, then error recording stops and z/OS continues with no error log recording. A daily or monthly procedure is used to copy logrec data set (SYSPLEX.LOGREC.ALLRECS or BC2A.LOGREC) into a daily OUTPUT ACCUMULATION DATA SET (IPOSAV.LOGREC) and to clear logrec data set (SYSPLEX.LOGREC.ALLRECS or BC2A.LOGREC) to be ready for new error recording followed by printing a ‘SYSTEM SUMMARY’ REPORT for analyzes. The OUTPUT ACCUMULATION DATA SET (IPOSAV.LOGREC) is copied into a weekly ACCUMULATION DATA SET (IPOSAV.LOGRECW) that may be saved on tape back-ups at regular intervals. To prevent the stop of error recording, go to the article “ZOS Logrec (Error log) processing” where the regular daily/monthly procedure for Logrec processing is described.
Incoming search terms:
- TSO RC=0008
Comments
No comments yet, be the first to add one!