IBM z/OS MVS Spooling : a Brief Introduction

IBM z/OS MVS Spooling : a Brief Introduction —

Spooling means a holding area on disk is used for input jobs waiting to run and output waiting to print.

This is a brief introduction to IBM z/OS MVS mainframe spooling.

The holding area is called spool space.  The imagery of spooling was probably taken from processes that wind some material such as thread, string, fabric or gift wrapping paper onto a spindle or spool.  In effect, spooling is a near-synonym for queueing.  Those paper towels on that roll in the kitchen are queued up waiting for you to use each one in turn, just like a report waiting to be printed or a set of JCL statements waiting for its turn to run.

On very early mainframe computers, the system read input jobs from a punched card reader, one line at a time, one line from each punched card.  It wrote printed output to a line printer, one line at a time.  Compared to disks – even the slower disks used decades ago – the card readers and line printers were super slow.  Bottlenecks, it might be said.  The system paused its other processing and waited while the next card was read, or while the next line was printed.  So that methodology was pretty well doomed.  It was okay as a first pass at getting a system to run — way better than an abacus — but that mega bottleneck had to go.  Hence came spooling.

HASP, Houston Automatic Spooling Priority program (system, subsystem) was early spooling software that was used with OS/360 (the ancestral precursor of z/OS MVS).  (See HASP origin story, if interested.)  HASP was the basis of development for JES2, which today is the most widely used spooling subsystem for z/OS MVS systems.  Another fairly widely used current spooling sub-system is JES3, based on an alternate early system called ASP.  We will focus on JES2 in this article because it is more widely used.   JES stands for Job Entry Subsystem.  In fact JES subsystems oversee both job entry (input) and processing of sysout (SYStem OUTput). 

Besides simply queueing the input and output, the spooling subsystem schedules it.  The details of the scheduling form the main point of interest for most of us. Preliminary to that, we might want to know a little about the basic pieces involved.

The Basic pieces

There are input classes, also called job classes, that control scheduling and resource limits

There are output classes, also called sysout classes, that control output print

There are real physical devices (few card readers, but many variations of printers and vaguely printer-like devices)

There are virtual devices. One virtual device is the “internal reader” used for software-submitted jobs, such as those sent in using the TSO submit command or FTP.  Virtual output devices include “external writers”.  An external writer is a program that reads and processes sysout files, and such a program can route the output to any available destination.  Many sysout files are never really printed, but are viewed (and further processed) directly from the spool space under TSO using a software product like SDSF.

There is spool sharing.  A JES2 spool space on disk (shared disk, called shared DASD) can be shared between two or more z/OS MVS systems with JES2 (with a current limit of 32 systems connected this way).  Each such system has a copy of JES2 running. Together they form a multi-access spool configuration (MAS).  Each JES2 subsystem sharing the same spool space can start jobs  from the waiting input queues on the shared spool, and can also select and process output from the shared spool.

There is checkpointing. This is obviously especially necessary when spool sharing is in use.

There is routing.  Again, useful with spool sharing, to enable you to route your job to run on a particular system, but also useful just to route your job’s output print files to print on a particular printer.

There are separate JES2 operator commands that the system operator can use to control the spooling subsystem, for example to change what classes of sysout can be sent to a specific printer, or what job classes are to be processed.  (These are the operator commands that start with a dollar sign $, or some alternative currency symbol depending on where your system is located.)

There is a set of very JCL-like control statements you can use to specify your requirements to the spooling subsystem.  (Sometimes called JECL, for Job Entry Control Language, as distinct from plain JCL, Job Control Language.)  For JES3, these statements begin with //* just like an ordinary JCL comment, so a job that has been running on a JES3 system can be copied to a system without JES3 and the JES3-specific JECL statements will simply be ignored as comments.  For JES2, on which we will focus here, the statements generally begin with /* in columns 1 and 2.  Common examples you may have seen are /*ROUTE and /*OUTPUT but notice that the newer OUTPUT statement in JCL is an upgrade from /*OUTPUT and the new OUTPUT statement offers more (and, well, newer) options.  Though the OUTPUT statement is newish, it is over a decade old, so you probably do have it on your system.

There are actual JCL parameters and statements that interact with JES2, such as the OUTPUT parameter on the DD statement, and the just-mentioned OUTPUT statement itself, which is pointed to by the parameter on the DD. 

Another example is the CLASS parameter on the JOB statement, which is used to designate the job class for job scheduling and execution.  The meanings of the individual job classes are totally made up for each site.  Some small development company might have just one job class for everything.  Big companies typically create complicated sets of job classes, each class defined with its own limits for resources such as execution time, region size, even the time of day when the jobs in each class are allowed to run.  Your site can define how many jobs of the same class are allowed to run concurrently, and the scheduling selection priority of each class relative to each other class.  Sometimes sites will set up informal rules which are not enforced by the software, but by local working rules, so that everyone there is presumed to know that they are only allowed to specify, for example, CLASS=E for emergency jobs.   (That’s one I happened to see someplace.)  If you want to know what job CLASS to specify for your various work, your best bet is to ask your co-workers, the people who are responsible for setting up the job classes, or some other knowledgeable source at your company.  Remember you can be held accountable for following rules you know nothing about that are not enforced by any software configuration, so don’t try to figure it out on your own, ask colleagues and other appropriate individuals what is permissible and expected.  Not joking.  JES2 Init & Tuning (Guide and reference) are the books that define how JES2 job classes have been configured, if you’re just curious to get a general idea of what the parameters are. The JES2 proc in proclib usually contains a HASPPARM DD statement pointing to where to find the JES2 configuration parameters on any particular system.  

In some cases similar considerations can apply for the use of SYSOUT print classes and the routing of such output to go to particular printers or to be printed at particular times.  The SYSOUT classes, like JOB classes, are entirely arbitrary and chosen by the responsible personnel at each site.  

MSGCLASS on the JOB statement controls where the job log goes — the JCL and messages portion of your listing.  The values you can specify for MSGCLASS are exactly the same as those for SYSOUT (whatever way that may be set up at your site).  If you want all your SYSOUT to go to the same place, along with your JCL and messages, specify that class as the value for MSGCLASS= on your job statement, and then specify SYSOUT=* on all of the DD statements for printed output files.  (That is, specify an asterisk as the value for SYSOUT= on the DD statements.)  

In many places, SYSOUT class A indicates real physical printed output on any printer, class X indicates routing to a held queue where it can be viewed from SDSF, and class Z specifies that the output immediately vanishes (Yup, that's an option).  However, there is no way to know for sure the details of how classes are set up at your particular site unless you ask about it.

Sometimes places maintain “secret” classes for specifying higher priority print jobs, or jobs that go to particular special reserved printers, and the secrets don’t stay secret of course.  Just because you see someone else using some print class, don’t assume it means it’s okay for you to use it for any particular job.  Ask around about the local rules and expectations.

So, for MSGCLASS (aka SYSOUT classes), as for JOB classes, the best thing is to ask whoever sets up the classes at your site; or, if that isn't practical, ask people working in the same area as you are, or just whoever you think is probably knowledgeable about the local setup.  Classes are set up by your site, for your site. 

An example of a JES2-related JCL statement that you have probably not yet seen is introduced with z/OS 2.2 — the JOBGROUP  statement, and an entire set of associated statements (ENDGROUP, SCHEDULE, BEFORE, AFTER, CONCURRENT), there are about ten of them – but that would be a topic for a follow-on post.  You probably don’t have z/OS 2.2 yet anyway, but it can be fun to know what’s coming.  JOBGROUP is coming.

That’s probably enough for an overview basic introduction.

The idea for this post came from a suggestion by Ian Watson.

 

References and Further Reading

z/OS concepts: JES2 compared to JES3
https://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zconcepts/zconc_jes2vsjes3.htm

 
z/OS JES2 Initialization and Tuning Guide, SA32-0991-00
How to initialize JES2 in a multi-access SPOOL configuration
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.hasa300/himas.htm
 
z/OS MVS JCL Reference (z/OS 2.2)
JES2 Execution Control Statements (This is where you can see the new JOBGROUP)
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.2.0/com.ibm.zos.v2r2.ieab600/jes2zone.htm
 
z/OS MVS JCL Reference, SA23-1385-00  (z/OS 2.1)
JES2 control statements
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieab600/j2st.htm
 
OUTPUT JCL statement
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieab600/outst.htm

 
z/OS JES2 Initialization and Tuning Reference, SA32-0992-00
Parameter description for JOBCLASS(class…|STC|TSU)
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.hasa400/has2u600106.htm

 
z/OS JES2 Initialization and Tuning Guide, SA32-0991-00
Defining the data set for JES2 initialization parameters
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.hasa300/defiset.htm

IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling IBM z/OS MVS spooling

Program Search Order

z/OS MVS Program Search Order

When you want to run a program or a TSO command, where does the system find it?  If there are multiple copies, how does it decide which one to use?  That is today’s topic.

The system searches for the item you require; It has a list of places to search; it gives you the first copy it finds.  The order in which it searches the list is called, unsurprisingly, search order

The search order is different depending on where you are when the request is made (where your task is running within the computer system, not where you're sitting geographically).   It also depends on what you request (program, JCL PROC, CLIST, REXX, etc). 

Types of searches: Normally we think of the basic search for an executable program. The search for TSO commands is almost identical to the search for programs executed in batch, with some extra bells and whistles.  There is different handling for a PROC request in JCL (since the search needs to cover JCL libraries, not program libraries).  The search is different when you request an exec in TSO by prefixing the name of the exec with a percent sign (%) — the percent sign signals the system to bypass searching for programs and go directly to searching for CLIST or REXX execs. Transaction systems such as CICS and IMS are again different: They use look-up tables set up for your CICS or IMS configuration.

Right now we’re only going to cover batch programs and TSO commands.

Overview of basic search order for executable programs:

  1. Command tables are consulted for special directions
  2. Programs already in storage within your area
  3. TASKLIB   (ISPLUSR,  LIBDEF,  ISPLLIB, TSOLIB) 
  4. STEPLIB if it exists
  5. JOBLIB only if there is no STEPLIB
  6. LPA (Link Pack Area)
  7. LINKLIST (System Libraries)
  8. CLIST and REXX only if running TSO

Let’s pretend you don’t believe that bit about STEPLIB and JOBLIB, that the system searches one or the other but not both.  Look here for verification:  Numbered item 3 in Section “Search order the system uses for programs” in the IBM publication “z/OS MVS Initialization and Tuning Guide” at this link: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieae100/ieae10018.htm

Batch Programs vs TSO commands

As noted, these two things are almost exactly the same.

You can type the name of a program when in READY mode in TSO, or in ISPF option 6, which mimics READY mode.  (Using ISPF 6 is similar to opening a DOS window on the PC.)  Under ISPF you can also enter the name of a program directly on any ISPF command line if you preface the program name with the word TSO followed by a space. 

So you can type the word IEBGENER when in READY mode, or you can put in “TSO  IEBGENER“ on the ISPF command line, and the system will fetch the same IEBGENER utility program just the same as it does when you say
“//  EXEC  PGM=IEBGENER” in batch JCL. 

There is a catch to this: the PARM you use in JCL is not mirrored when you invoke a program under TSO this way.  If you enter any parameters on the command line in TSO, they are passed to the program in a different format than a PARM.   When you type the name of a program to invoke it under TSO, what usually happens is that the program is started, but instead of getting a pointer it can use to find a PARM string, the program receives a pointer that can be used to find the TSO line command that was entered.  The two formats are similar enough so that a program expecting to get a PARM will be confused by the CPPL pointer (CPPL = Command Processor Parameter List).  Typically such a program will issue a message saying that the PARM parameters are invalid.

So, Let's look at how the system searches for the program.

Optional ISPF command tables (TSO/ISPF only) 

We mention command tables first because the command tables are the first thing the system checks.  Fortunately the tables are optional.  Unfortunately they can cause problems.  Fortunately that is rare.  So It’s fine for you to skip ahead to the next section, you won’t miss much; just be aware that the command tables exist, so you can remember it if you ever need to diagnose a quirky problem in this area.

Under TSO/ISPF, ISPF-specific command tables can be used.  These can alter where the system searches for commands named in such a table.  This is an area where IBM makes changes from time to time, so if you need to set up these tables, consult the IBM documentation for your exact release level. 

There is a basic general ISPF TSO command table called ISPTCM that does not vary depending on the ISPF application.

Also, search order can vary within ISPF depending on the ISPF application. 

For example, when you go into a product like File-AID, the search order might be changed to use the command tables specific to that application. 

So there are also command tables specific to the ISPF application that is running.  In addition to a general Application Command Table, there can be up to 3 site command tables and up to 3 user command tables, plus a system command table.  Within these, the search order can vary depending on what is specified at your site during setup. 

If you are having a problem related to command tables, or if you need to set one up, consult the IBM references.   For z/OS v2r2, see "Customizing the ISPF TSO command table (ISPTCM)" and “Customizing command tables In the IBM manual "ISPF Planning and Customizing".

ISPF command table(s) can influence other things besides search order, including:

(a) A table can specify that a particular program is to run in APF-authorized mode, allowing the program to do privileged processes (APF is an entirely separate topic and not covered herein).

(b) A table can specify that a particular program must be obtained from LPA only (bypassing the normal search order).  (We’ll introduce LPA further down.)

(c) A table can change characteristics of how a command is executed.  For example the table can specify that any given program must always be invoked as a PARM-processing ordinary program, or the table can specify instead that the program must always be invoked as a TSO command that accepts parameters entered on the command line along with the command name, e.g. LISTDS ‘some.dataset.name’

Programs are not required to be listed in any of these command tables.  If a program is not listed in any special table, then default characteristics are applied to the program, and the module itself is located via the normal search order, which is basically the same as for executable programs in batch. 

Most notably, a command table can cause problems if an old version exists but that fact is not known to the current local Systems people (z/OS System Admins are not called Admins as they might be on lesser computer systems; on the mainframe they're called "System Programmers" generally, or some more specialized title).  Such a table might prevent or warp the execution of a program because the characteristics of a named program have changed from one release to another.

If a command is defined in a command table, but the program that the system finds does not conform to the demands imposed by the settings in the table, that situation can be the cause of quirky problems, including the “COMMAND NOT FOUND” message.  Yes, that might perhaps be IBM’s idea of a practical joke, but it happens like this: If the table indicates that a program has to be found in some special place, but the system finds the program in the wrong place, then in some cases you get the COMMAND NOT FOUND message.  I know, you’re cracking up laughing.  That’s the effect it seems to have on most people when they run into it.  Not everyone, of course.

Other subsystems such as IMS and CICS also use command tables, and these are not covered by the current discussion.  Consult IMS and CICS documentation for specifics of those subsystems. 

Programs already loaded into your memory

Before STEPLIB or anything like that, the system searches amongst the programs that have already been loaded into your job’s memory.  Your TSO session counts as a job.  

There can be multiple tasks running within your job – Think split screens under TSO/ISPF.  In that case the system first searches the program modules that belong to the task where the request was issued, and after that it searches those that belong to the job itself.  Relevant jargon : a task has a Load List, which has Load List Elements (LLE); the job itself has Job pack area (JPA).

If a copy of the module is found in your job’s memory, in either the task's LLE or the job's JPA, that copy of the module will be used if it is available.  

If the module is marked as reentrant – if the program has a flag set to promise that it does not modify itself at all – then any number of tasks can use the same copy simultaneously, so you always get it. If the module is not marked as reentrant, but it is marked as serially reusable, that means the program can modify itself while it's running as long as, at the end, it puts everything back the way it was originally; in that case a second task can use an existing copy of the module after the previous task finishes.  If neither of those flags are set, then the system has to load a fresh copy of the load module every time it is to be run.

If the module is not reentrant and it is already in use, as often happens in multi-user transaction processing subsystems like IMS and CICS, the system might make you wait until the previous caller finishes, or else it might load an additional copy of the module.  In CICS and IMS this depends on CICS/IMS configuration parameters.  In ordinary jobs and TSO sessions, the system normally just loads a new copy.

Note that if a usable copy of a module is found already loaded in memory, either in the task's LLE or in the job's JPA, that copy will be used EVEN if your program has specified a DCB on the LOAD macro to tell the system to load the module from some other library (unless the copy in memory is "not reusable", and then a new copy will be obtained).  Hunh?  Yeah, suppose your program, while it is running, asks to load another program, specifying that the other program is to be loaded from some specific Library.  Quelle surprise, if there is already a copy of the module sitting in memory in your area, then the LOAD does not go look in that other library.

For further nuances on this, if it is a topic of concern for you, see the IBM write-up under the topic "The Search for the Load Module" in the z/OS MVS Assembler Services Guide

TASKLIB 

Most jobs do not use TASKLIB.  TSO does.  

Essentially TASKLIB works like a stand-in for STEPLIB.  A job can’t just reallocate STEPLIB once the job is already running.  For batch jobs the situation doesn’t come up much.  Under TSO, people often find cases where they’d like to be able to reallocate STEPLIB.  Enter TASKLIB.  STEPLIB stays where it is, but TASKLIB can sneak in ahead of it.

Under TSO, ISPF uses the ddname ISPLLIB as its main TASKLIB.

The ddname ISPLUSR exists so that individual users – you yourself for example – can use their own private load libraries, and tell ISPF to adopt those libraries as part of the TASKLIB, in fact at the top of the list.  When ISPF starts, it checks to see if the ddname ISPLUSR is allocated.  If it is, then ISPF assigns TASKLIB with ISPLUSR first, followed by ISPLLIB.  As long as you allocate ISPLUSR before you start ISPF, then ISPLUSR will be searched before ISPLLIB.  

In fact that USR suffix was invented just for such situations.  It’s a tiny digression, but ISPF allows you to assign ISPPUSR to pre-empt ISPPLIB for ISPF screens (Panels), and so on for other ISP*LIB ddnames; they can be pre-empted by assigning your own libraries to their ISP*USR counterparts. 

Up to 15 datasets can be concatenated on ISPLUSR.  If you allocate more, only the first 15 will be searched.

If you allocate ISPLUSR, you have to do it before you start ISPF, and it applies for the duration of the ISPF session.

Not so with LIBDEF.  The LIBDEF command can be used while ISPF is active.  Datasets you LIBDEF onto ISPLLIB are searched before other ISPLLIB datasets, but after ISPLUSR datasets.

The TSOLIB command can also be used to add datasets onto TASKLIB

The TSOLIB command, if used, must be invoked from READY mode prior to going into ISPF.   

Why yet another way of putting datasets onto TASKLIB?  The other three methods just discussed are specific to ISPF. The TSOLIB command is applicable to READY mode also. It can be used even when ISPF is not active.  For programs that run under TSO but not under ISPF, the only TASKLIB datasets used are those activated by TSOLIB.  Within ISPF, any dataset you add to TASKLIB by using "TSOLIB ACTIVATE" will come last within TASKLIB, after ISPLLIB.  

Also, TASKLIB load libraries activated by the TSOLIB command are available to non-ISPF modules called by ISPF-enabled programs.  For example, if a program running under ISPF calls something like IEBCOPY, and then IEBCOPY itself issues a LOAD to get some other module it wants to call, do not expect the system to look in ISPLLIB for the module that IEBCOPY is trying to load.  It should check TSOLIB, though.  However, some programs bypass TASKLIB search altogether.

Under ISPF, This is the search order within TASKLIB:

    ISPLUSR
    LIBDEF
    ISPLLIB
    TSOLIB

For non-ISPF TSO, only TSOLIB is used for TASKLIB.

To find out your LIBDEF allocations, enter ISPLIBD on the ISPF command line. (Not TSO ISPLIBD, just plain ISPLIBD)

TASKLIB with the CALL command in TSO

When you use CALL to run a program under TSO, the library name on the CALL command becomes a TASKLIB during the time the called program is running.  CALL is often used within CLIST and REXX execs, even though you may not use CALL much yourself directly.

So if you say CALL 'SYS1.LINKLIB(IEBGENER)' from ISPF option 6 or from a CLIST or REXX, then 'SYS1.LINKLIB' will be used to satisfy other LOAD requests that might be issued by IEBGENER or its subtasks while the called IEBGENER is running.  Ah, yes, the system is full of nuances and special cases like this one; I'm just giving you the highlights.  This entire article is somewhat of a simplification.  What joys, you might be thinking, must await you in the world of mainframes.

STEPLIB or JOBLIB

If STEPLIB and JOBLIB are both in the JCL, the STEPLIB is used and the JOBLIB is ignored.

The system does NOT search both.

LPA (Link Pack Area)

You know that there are default system load libraries that are used when you don’t have STEPLIB or JOBLIB, or when your STEPLIB or JOBLIB does not contain the program to be executed.  

The original list of the names of those system load libraries is called LINKLIST.  The first original system load library was called SYS1.LINKLIB, and when they wanted to have more than one system library they came up with the name LINKLIST to designate the list of library names that were to be treated as logical extensions of LINKLIB.

The drawback with LINKLIST, as with STEPLIB and JOBLIB, is that when you request a program from them, the system actually has to go and read the program into memory from disk.  That’s overhead.  So they invented LPA (which stands for Link Pack Area — go figure.).

Some programs are used so often by so many jobs that it makes sense to keep them loaded into memory permanently.  Well, virtual memory.  As long as such a program is not self-modifying, multiple jobs can use the same copy at the same time:  Additional savings.  So an area of memory is reserved for LPA.  Modules from SYS1.LPALIB (and its concatenation partners) are loaded into LPA.  It is pageable, so more heavily used modules replace less used modules dynamically.

Sounds good, but more tweaks came.  Some places consider some programs so important and so response-time-sensitive that they want those programs to be kept in memory all the time, even if they haven’t been used for a few minutes.  And so on, until we now have several subsets of LPA.

Within LPA, the following search order applies:

Dynamic LPA, from the list in ‘SYS1.PARMLIB(PROG**)’

Fixed LPA (FLPA), from the list in ‘SYS1.PARMLIB(IEAFIX**)’

Modified LPA (MLPA), from the list in ‘SYS1.PARMLIB(IEALPA**)’

Pageable LPA (PLPA), from the list in (LPALST**) and/or (PROG**)

LINKLIST

 LINKLIST Libraries are specified using SYS1.PARMLIB(PROG**) and/or (LNKLST**).

SYS1.LINKLIB is included in the list of System Libraries even if it is not named explicitly. 

An overview of LINKLIB was just given in the introduction to LPA, so you know this already, or at least you can refer back to it above if you skipped ahead.

Note that LINKLIST libraries are controlled by LLA; whenever any LINKLIST module is updated the system's BLDL needs to be rebuilt by an LLA refresh.  If LLA is not refreshed the old version of the module will continue to be given to anyone requesting that module.

What’s LLA, you might ask.  For any library under LLA control, the system reads the directory of each library, and keeps the directory permanently in memory.  A library directory contains the disk address of every library member.  Hence keeping the directory in memory considerably speeds up the process of finding any member.  It speeds that up more than one might think, because PDS directory blocks have a very small block size, 256 bytes, and these days the directories of production load libraries can contain a lot of members, two facts which taken together mean that reading an entire PDS directory from disk can require many READ operations and hence be time-consuming.  If you repeat that delay for almost every program search in every job, you have a drag on the system.  So LLA gives a meaningful performance improvement for reading members from PDS-type libraries.  For PDSE libraries too, but for different reasons; PDSE libraries do not have directory blocks like ordinary PDS libraries.  Anyway, the price you pay for the improvement is that the in-memory copies of the directories have to be rebuilt whenever a directory is updated, that is, whenever a library member is replaced.  What does LLA stand for?  Originally it stood for LINKLIST LookAside, but when the concept was extended to cover other libraries besides LINKLIST the name was changed to Library LookAside.

Under TSO, CLIST and REXX execs

Under TSO, if a module is not found in any of the above places, there is a final search for a CLIST or REXX exec matching the requested name. 

CLIST and REXX execs are obtained by searching ddnames SYSUEXEC, SYSUPROC, SYSEXEC, and SYSPROC (in that order, if the order has not been deliberately changed).  Note that SYSUEXEC and SYSEXEC are for REXX members only, whereas SYSPROC and SYSUPROC can contain both REXX and CLIST members, with the proviso that a REXX member residing in the SYSPROC family of libraries must contain the word REXX on the first line, typically as a comment /* REXX */

These four ddnames can be changed and other ddnames can be added by use of the ALTLIB command.  Also the order of search within these ddnames can be altered with the ALTLIB command (among other ways)

.SYSPROC was the original CLIST ddname.  A CLIST library can also include REXX execs. The SYSEXEC ddname was added to be used just for REXX.  In the spirit of ISPLUSR et al, a USER version of each was added, called SYS­UPROC and SYSUEXEC.

The default REXX library ddname SYSEXEC can be changed to something other than SYSEXEC by MVS system installation parameters.

Prefixing the TSO command name with the percent sign (%) causes the system to skip directly to the CLIST and REXX part of the search, rather than looking every other place first. 

To find the actual search order in effect within the CLIST and EXEC ddnames for your TSO session at any given time, use the command TSO ALTLIB DISPLAY. 

That's it for program search order.  It's a simplification of course. ;-)

 

References, Further reading

z/OS Basic Skills, Mainframe concepts, z/OS system installation and maintenance
Search order for programs
http://www.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zsysprog/zsysprogc_searchorder.htm

z/OS TSO/E REXX Reference, SA32-0972-00 
Using SYSPROC and SYSEXEC for REXX execs
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ikja300/dup0003.htm

z/OS V2R2 ISPF Services Guide 
Application data element search order
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.2.0/com.ibm.zos.v2r2.f54sg00/aso.htm

z/OS ISPF Services Guide, SC19-3626-00 
LIBDEF—allocate application libraries
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.f54sg00/libdef.htm

Numbered item 3 in Section “Search order the system uses for programs” in the IBM publication “z/OS MVS Initialization and Tuning Guide” at this link: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieae100/ieae10018.htm

"The Search for the Load Module" in the z/OS MVS Assembler Services Guide

"Customizing the ISPF TSO command table (ISPTCM)" and “Customizing command tables In the IBM manual "ISPF Planning and Customizing"