CLIST developer on retainer: JSUBPAY ALLOCATE DISP(EXCL) on PAYROLL.BATCH.CONTROL holds dataset enqueue through error exit without FREE — PAYROLL2 gets IKJ56231I ALREADY IN USE

z/OS 2.4 • TSO/E 4.4 • CLIST • ALLOCATE DISP(EXCL) • ISRDDN ENQ • D GRS,CONTENTION • retainer billing

At Midwest Federal Savings Bank, the payroll processing team ran two TSO user profiles — PAYROLL1 and PAYROLL2 — so that two payroll analysts could submit the biweekly and supplemental payroll batch jobs independently. The system worked fine for years. Then one October morning PAYROLL1’s job submission failed partway through — JES2 was at its hold threshold and rejected the submit — and PAYROLL2, who needed to submit the supplemental payroll run, got a message she had never seen before: IKJ56231I DATA SET PAYROLL.BATCH.CONTROL ALREADY IN USE, RETRY LATER.

The dataset was still exclusively allocated by PAYROLL1’s TSO session. PAYROLL1 had logged off after the error. The session ended. But the dataset enqueue — a z/OS Global Resource Serialization (GRS) SYSDSN EXCL lock — stayed held for four hours and twenty-two minutes until a systems programmer intervened. The CLIST retainer consultant was called the same afternoon.

The JSUBPAY CLIST and the ALLOCATE without FREE

JSUBPAY is the payroll batch submission CLIST, stored in library PAYROLL.TSO.CLIST. When an analyst runs it from TSO, it performs three steps: allocate the job control dataset exclusively, write the JES2 job card and pay period parameters, then SUBMIT. The relevant section:

PROC 2 PAYDATE CYCLE
CONTROL NOMSG NOLIST
*
ALLOCATE DATASET('PAYROLL.BATCH.CONTROL') DISP(EXCL) SHR REUSE
IF &LASTCC NE 0 THEN GOTO ALLOCERR
*
OPENFILE PAYCTLDD OUTPUT
PUTFILE PAYCTLDD  &STR(//PAYRLRUN JOB (ACCT),'PAYROLL',CLASS=A,MSGLEVEL=(1,1))
PUTFILE PAYCTLDD  &STR(//  EXEC PGM=PAYMAINP)
PUTFILE PAYCTLDD  &STR(PAYDATE=&PAYDATE)
PUTFILE PAYCTLDD  &STR(CYCLE=&CYCLE)
CLOSFILE PAYCTLDD
*
SUBMIT 'PAYROLL.BATCH.CONTROL'
IF &LASTCC NE 0 THEN GOTO SUBMITERR   /* ← no FREE before this GOTO */
*
FREE DATASET('PAYROLL.BATCH.CONTROL')
WRITE JOB SUBMITTED SUCCESSFULLY
EXIT
*
ALLOCERR: WRITE ALLOCATE FAILED RC=&LASTCC
EXIT                                   /* ← no FREE here (ALLOC failed so none to free) */
*
SUBMITERR: WRITE SUBMIT FAILED RC=&LASTCC
EXIT                                   /* ← FREE is missing here */
END PROC

The bug is on the SUBMITERR exit path. ALLOCATE DATASET('PAYROLL.BATCH.CONTROL') DISP(EXCL) succeeded — the SYSDSN EXCL enqueue was acquired. The subsequent PUTFILE writes succeeded. The SUBMIT failed (JES2 RC=12, job entry subsystem hold threshold). Control branched to SUBMITERR, printed the error message, and exited. The FREE DATASET('PAYROLL.BATCH.CONTROL') line that exists on the success path was never reached. The TSO ALLOCATE command holds a dataset allocation — and its associated GRS EXCL enqueue — for the lifetime of the TSO session or until a FREE is issued. PAYROLL1 then logged off, but the TSO session had already ended with the dataset still allocated. z/OS held the enqueue for a configurable cleanup interval; on this system it was set to match the TSO session timeout, so the enqueue stayed for hours.

ISRDDN and D GRS,CONTENTION diagnosis

The retainer consultant logged in under a systems programmer profile and ran the ISRDDN command in ISPF. The Dataset Allocations display showed PAYROLL.BATCH.CONTROL with disposition EXCL, allocated to TSO session TSOPAY1 under user PAYROLL1 — even though PAYROLL1 appeared logged off. The allocation was a ghost from the abandoned session.

At the operator console, the command D GRS,CONTENTION confirmed:

ISG020I RESOURCE CONTENTION:
  SYSDSN  PAYROLL.BATCH.CONTROL
  OWN  TSOPAY1  EXCL
  WAIT TSOPAY2  EXCL   (1 WAITER)

PAYROLL2’s JSUBPAY CLIST was waiting at its own ALLOCATE DISP(EXCL) call, blocked behind the ghost enqueue from PAYROLL1. The systems programmer issued a VARY GRS,RELEASE,RESOURCE=(SYSDSN,PAYROLL.BATCH.CONTROL) to force-release the enqueue. PAYROLL2’s ALLOCATE immediately succeeded and the supplemental payroll job was submitted — four hours late.

Fix: FREE before every EXIT and GOTO on error paths

The fix required adding FREE DATASET('PAYROLL.BATCH.CONTROL') before every exit path that follows a successful ALLOCATE. The corrected CLIST:

PROC 2 PAYDATE CYCLE
CONTROL MAIN NOFLUSH                   /* don't abort on TSO error before cleanup */
*
ALLOCATE DATASET('PAYROLL.BATCH.CONTROL') DISP(EXCL) SHR REUSE
IF &LASTCC NE 0 THEN GOTO ALLOCERR    /* alloc failed — nothing to FREE */
SET &ALLOCATED = 1                     /* flag: ALLOCATE succeeded */
*
OPENFILE PAYCTLDD OUTPUT
PUTFILE PAYCTLDD  &STR(//PAYRLRUN JOB ...)
... (write parameters) ...
CLOSFILE PAYCTLDD
*
SUBMIT 'PAYROLL.BATCH.CONTROL'
IF &LASTCC NE 0 THEN GOTO SUBMITERR
*
FREE DATASET('PAYROLL.BATCH.CONTROL')  /* success path */
WRITE JOB SUBMITTED SUCCESSFULLY
EXIT
*
SUBMITERR: +
  FREE DATASET('PAYROLL.BATCH.CONTROL')   /* ← added */
  WRITE SUBMIT FAILED RC=&LASTCC
  EXIT
*
ALLOCERR: WRITE ALLOCATE FAILED RC=&LASTCC
EXIT
END PROC

Two additional hardening changes were applied: CONTROL MAIN NOFLUSH was added at the top so that if TSO itself raises an unexpected error during OPENFILE/PUTFILE/CLOSFILE, the CLIST continues executing rather than aborting — giving it a chance to reach a cleanup EXIT. An ATTN EXIT routine was also added to handle PA1 attention interrupts: if the analyst presses PA1 to abort mid-CLIST, the attention exit runs the FREE before terminating.

After the fix: ISRDDN showed zero EXCL enqueues on PAYROLL.BATCH.CONTROL after a test SUBMIT failure. D GRS,CONTENTION: no contention on the resource. IKJ56231I: never seen again in production.

How HourTab helps CLIST retainer consultants

The fix took 2 hours total. But the CLIST retainer consultant also spent 30 minutes walking the payroll manager through what a GRS enqueue is, why TSO ALLOCATE holds it, and why CONTROL MAIN NOFLUSH matters. That explanation time is invisible in a flat invoice but visible in a HourTab entry labeled “explanation + stakeholder walk-through.” The payroll manager appreciated knowing she was paying for education, not just keystrokes. She bookmarked the HourTab link and checked it herself before the next quarterly billing review.

HourTab turns a time-tracker CSV into a public retainer-hours URL your client can bookmark — no login, no portal, no “how many hours do I have left?” emails. Free for one retainer. $9/mo for up to ten.

FAQ: z/OS TSO CLIST developer retainers

See also: REXX developer retainer · JCL developer retainer · CICS developer retainer