z/OS MVS Program Search Order
When you want to run a program or a TSO command, where does the system find it? If there are multiple copies, how does it decide which one to use? That is today’s topic.
The system searches for the item you require; It has a list of places to search; it gives you the first copy it finds. The order in which it searches the list is called, unsurprisingly, search order.
The search order is different depending on where you are when the request is made (where your task is running within the computer system, not where you're sitting geographically). It also depends on what you request (program, JCL PROC, CLIST, REXX, etc).
Types of searches: Normally we think of the basic search for an executable program. The search for TSO commands is almost identical to the search for programs executed in batch, with some extra bells and whistles. There is different handling for a PROC request in JCL (since the search needs to cover JCL libraries, not program libraries). The search is different when you request an exec in TSO by prefixing the name of the exec with a percent sign (%) — the percent sign signals the system to bypass searching for programs and go directly to searching for CLIST or REXX execs. Transaction systems such as CICS and IMS are again different: They use look-up tables set up for your CICS or IMS configuration.
Right now we’re only going to cover batch programs and TSO commands.
Overview of basic search order for executable programs:
- Command tables are consulted for special directions
- Programs already in storage within your area
- TASKLIB (ISPLUSR, LIBDEF, ISPLLIB, TSOLIB)
- STEPLIB if it exists
- JOBLIB only if there is no STEPLIB
- LPA (Link Pack Area)
- LINKLIST (System Libraries)
- CLIST and REXX only if running TSO
Let’s pretend you don’t believe that bit about STEPLIB and JOBLIB, that the system searches one or the other but not both. Look here for verification: Numbered item 3 in Section “Search order the system uses for programs” in the IBM publication “z/OS MVS Initialization and Tuning Guide” at this link: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieae100/ieae10018.htm
Batch Programs vs TSO commands
As noted, these two things are almost exactly the same.
You can type the name of a program when in READY mode in TSO, or in ISPF option 6, which mimics READY mode. (Using ISPF 6 is similar to opening a DOS window on the PC.) Under ISPF you can also enter the name of a program directly on any ISPF command line if you preface the program name with the word TSO followed by a space.
So you can type the word IEBGENER when in READY mode, or you can put in “TSO IEBGENER“ on the ISPF command line, and the system will fetch the same IEBGENER utility program just the same as it does when you say
“// EXEC PGM=IEBGENER” in batch JCL.
There is a catch to this: the PARM you use in JCL is not mirrored when you invoke a program under TSO this way. If you enter any parameters on the command line in TSO, they are passed to the program in a different format than a PARM. When you type the name of a program to invoke it under TSO, what usually happens is that the program is started, but instead of getting a pointer it can use to find a PARM string, the program receives a pointer that can be used to find the TSO line command that was entered. The two formats are similar enough so that a program expecting to get a PARM will be confused by the CPPL pointer (CPPL = Command Processor Parameter List). Typically such a program will issue a message saying that the PARM parameters are invalid.
So, Let's look at how the system searches for the program.
Optional ISPF command tables (TSO/ISPF only)
We mention command tables first because the command tables are the first thing the system checks. Fortunately the tables are optional. Unfortunately they can cause problems. Fortunately that is rare. So It’s fine for you to skip ahead to the next section, you won’t miss much; just be aware that the command tables exist, so you can remember it if you ever need to diagnose a quirky problem in this area.
Under TSO/ISPF, ISPF-specific command tables can be used. These can alter where the system searches for commands named in such a table. This is an area where IBM makes changes from time to time, so if you need to set up these tables, consult the IBM documentation for your exact release level.
There is a basic general ISPF TSO command table called ISPTCM that does not vary depending on the ISPF application.
Also, search order can vary within ISPF depending on the ISPF application.
For example, when you go into a product like File-AID, the search order might be changed to use the command tables specific to that application.
So there are also command tables specific to the ISPF application that is running. In addition to a general Application Command Table, there can be up to 3 site command tables and up to 3 user command tables, plus a system command table. Within these, the search order can vary depending on what is specified at your site during setup.
If you are having a problem related to command tables, or if you need to set one up, consult the IBM references. For z/OS v2r2, see "Customizing the ISPF TSO command table (ISPTCM)" and “Customizing command tables” In the IBM manual "ISPF Planning and Customizing".
ISPF command table(s) can influence other things besides search order, including:
(a) A table can specify that a particular program is to run in APF-authorized mode, allowing the program to do privileged processes (APF is an entirely separate topic and not covered herein).
(b) A table can specify that a particular program must be obtained from LPA only (bypassing the normal search order). (We’ll introduce LPA further down.)
(c) A table can change characteristics of how a command is executed. For example the table can specify that any given program must always be invoked as a PARM-processing ordinary program, or the table can specify instead that the program must always be invoked as a TSO command that accepts parameters entered on the command line along with the command name, e.g. LISTDS ‘some.dataset.name’
Programs are not required to be listed in any of these command tables. If a program is not listed in any special table, then default characteristics are applied to the program, and the module itself is located via the normal search order, which is basically the same as for executable programs in batch.
Most notably, a command table can cause problems if an old version exists but that fact is not known to the current local Systems people (z/OS System Admins are not called Admins as they might be on lesser computer systems; on the mainframe they're called "System Programmers" generally, or some more specialized title). Such a table might prevent or warp the execution of a program because the characteristics of a named program have changed from one release to another.
If a command is defined in a command table, but the program that the system finds does not conform to the demands imposed by the settings in the table, that situation can be the cause of quirky problems, including the “COMMAND NOT FOUND” message. Yes, that might perhaps be IBM’s idea of a practical joke, but it happens like this: If the table indicates that a program has to be found in some special place, but the system finds the program in the wrong place, then in some cases you get the COMMAND NOT FOUND message. I know, you’re cracking up laughing. That’s the effect it seems to have on most people when they run into it. Not everyone, of course.
Other subsystems such as IMS and CICS also use command tables, and these are not covered by the current discussion. Consult IMS and CICS documentation for specifics of those subsystems.
Programs already loaded into your memory
Before STEPLIB or anything like that, the system searches amongst the programs that have already been loaded into your job’s memory. Your TSO session counts as a job.
There can be multiple tasks running within your job – Think split screens under TSO/ISPF. In that case the system first searches the program modules that belong to the task where the request was issued, and after that it searches those that belong to the job itself. Relevant jargon : a task has a Load List, which has Load List Elements (LLE); the job itself has Job pack area (JPA).
If a copy of the module is found in your job’s memory, in either the task's LLE or the job's JPA, that copy of the module will be used if it is available.
If the module is marked as reentrant – if the program has a flag set to promise that it does not modify itself at all – then any number of tasks can use the same copy simultaneously, so you always get it. If the module is not marked as reentrant, but it is marked as serially reusable, that means the program can modify itself while it's running as long as, at the end, it puts everything back the way it was originally; in that case a second task can use an existing copy of the module after the previous task finishes. If neither of those flags are set, then the system has to load a fresh copy of the load module every time it is to be run.
If the module is not reentrant and it is already in use, as often happens in multi-user transaction processing subsystems like IMS and CICS, the system might make you wait until the previous caller finishes, or else it might load an additional copy of the module. In CICS and IMS this depends on CICS/IMS configuration parameters. In ordinary jobs and TSO sessions, the system normally just loads a new copy.
Note that if a usable copy of a module is found already loaded in memory, either in the task's LLE or in the job's JPA, that copy will be used EVEN if your program has specified a DCB on the LOAD macro to tell the system to load the module from some other library (unless the copy in memory is "not reusable", and then a new copy will be obtained). Hunh? Yeah, suppose your program, while it is running, asks to load another program, specifying that the other program is to be loaded from some specific Library. Quelle surprise, if there is already a copy of the module sitting in memory in your area, then the LOAD does not go look in that other library.
For further nuances on this, if it is a topic of concern for you, see the IBM write-up under the topic "The Search for the Load Module" in the z/OS MVS Assembler Services Guide
Most jobs do not use TASKLIB. TSO does.
Essentially TASKLIB works like a stand-in for STEPLIB. A job can’t just reallocate STEPLIB once the job is already running. For batch jobs the situation doesn’t come up much. Under TSO, people often find cases where they’d like to be able to reallocate STEPLIB. Enter TASKLIB. STEPLIB stays where it is, but TASKLIB can sneak in ahead of it.
Under TSO, ISPF uses the ddname ISPLLIB as its main TASKLIB.
The ddname ISPLUSR exists so that individual users – you yourself for example – can use their own private load libraries, and tell ISPF to adopt those libraries as part of the TASKLIB, in fact at the top of the list. When ISPF starts, it checks to see if the ddname ISPLUSR is allocated. If it is, then ISPF assigns TASKLIB with ISPLUSR first, followed by ISPLLIB. As long as you allocate ISPLUSR before you start ISPF, then ISPLUSR will be searched before ISPLLIB.
In fact that USR suffix was invented just for such situations. It’s a tiny digression, but ISPF allows you to assign ISPPUSR to pre-empt ISPPLIB for ISPF screens (Panels), and so on for other ISP*LIB ddnames; they can be pre-empted by assigning your own libraries to their ISP*USR counterparts.
Up to 15 datasets can be concatenated on ISPLUSR. If you allocate more, only the first 15 will be searched.
If you allocate ISPLUSR, you have to do it before you start ISPF, and it applies for the duration of the ISPF session.
Not so with LIBDEF. The LIBDEF command can be used while ISPF is active. Datasets you LIBDEF onto ISPLLIB are searched before other ISPLLIB datasets, but after ISPLUSR datasets.
The TSOLIB command can also be used to add datasets onto TASKLIB.
The TSOLIB command, if used, must be invoked from READY mode prior to going into ISPF.
Why yet another way of putting datasets onto TASKLIB? The other three methods just discussed are specific to ISPF. The TSOLIB command is applicable to READY mode also. It can be used even when ISPF is not active. For programs that run under TSO but not under ISPF, the only TASKLIB datasets used are those activated by TSOLIB. Within ISPF, any dataset you add to TASKLIB by using "TSOLIB ACTIVATE" will come last within TASKLIB, after ISPLLIB.
Also, TASKLIB load libraries activated by the TSOLIB command are available to non-ISPF modules called by ISPF-enabled programs. For example, if a program running under ISPF calls something like IEBCOPY, and then IEBCOPY itself issues a LOAD to get some other module it wants to call, do not expect the system to look in ISPLLIB for the module that IEBCOPY is trying to load. It should check TSOLIB, though. However, some programs bypass TASKLIB search altogether.
Under ISPF, This is the search order within TASKLIB:
For non-ISPF TSO, only TSOLIB is used for TASKLIB.
To find out your LIBDEF allocations, enter ISPLIBD on the ISPF command line. (Not TSO ISPLIBD, just plain ISPLIBD)
TASKLIB with the CALL command in TSO
When you use CALL to run a program under TSO, the library name on the CALL command becomes a TASKLIB during the time the called program is running. CALL is often used within CLIST and REXX execs, even though you may not use CALL much yourself directly.
So if you say CALL 'SYS1.LINKLIB(IEBGENER)' from ISPF option 6 or from a CLIST or REXX, then 'SYS1.LINKLIB' will be used to satisfy other LOAD requests that might be issued by IEBGENER or its subtasks while the called IEBGENER is running. Ah, yes, the system is full of nuances and special cases like this one; I'm just giving you the highlights. This entire article is somewhat of a simplification. What joys, you might be thinking, must await you in the world of mainframes.
STEPLIB or JOBLIB
If STEPLIB and JOBLIB are both in the JCL, the STEPLIB is used and the JOBLIB is ignored.
The system does NOT search both.
LPA (Link Pack Area)
You know that there are default system load libraries that are used when you don’t have STEPLIB or JOBLIB, or when your STEPLIB or JOBLIB does not contain the program to be executed.
The original list of the names of those system load libraries is called LINKLIST. The first original system load library was called SYS1.LINKLIB, and when they wanted to have more than one system library they came up with the name LINKLIST to designate the list of library names that were to be treated as logical extensions of LINKLIB.
The drawback with LINKLIST, as with STEPLIB and JOBLIB, is that when you request a program from them, the system actually has to go and read the program into memory from disk. That’s overhead. So they invented LPA (which stands for Link Pack Area — go figure.).
Some programs are used so often by so many jobs that it makes sense to keep them loaded into memory permanently. Well, virtual memory. As long as such a program is not self-modifying, multiple jobs can use the same copy at the same time: Additional savings. So an area of memory is reserved for LPA. Modules from SYS1.LPALIB (and its concatenation partners) are loaded into LPA. It is pageable, so more heavily used modules replace less used modules dynamically.
Sounds good, but more tweaks came. Some places consider some programs so important and so response-time-sensitive that they want those programs to be kept in memory all the time, even if they haven’t been used for a few minutes. And so on, until we now have several subsets of LPA.
Within LPA, the following search order applies:
Dynamic LPA, from the list in ‘SYS1.PARMLIB(PROG**)’
Fixed LPA (FLPA), from the list in ‘SYS1.PARMLIB(IEAFIX**)’
Modified LPA (MLPA), from the list in ‘SYS1.PARMLIB(IEALPA**)’
Pageable LPA (PLPA), from the list in (LPALST**) and/or (PROG**)
LINKLIST Libraries are specified using SYS1.PARMLIB(PROG**) and/or (LNKLST**).
SYS1.LINKLIB is included in the list of System Libraries even if it is not named explicitly.
An overview of LINKLIB was just given in the introduction to LPA, so you know this already, or at least you can refer back to it above if you skipped ahead.
Note that LINKLIST libraries are controlled by LLA; whenever any LINKLIST module is updated the system's BLDL needs to be rebuilt by an LLA refresh. If LLA is not refreshed the old version of the module will continue to be given to anyone requesting that module.
What’s LLA, you might ask. For any library under LLA control, the system reads the directory of each library, and keeps the directory permanently in memory. A library directory contains the disk address of every library member. Hence keeping the directory in memory considerably speeds up the process of finding any member. It speeds that up more than one might think, because PDS directory blocks have a very small block size, 256 bytes, and these days the directories of production load libraries can contain a lot of members, two facts which taken together mean that reading an entire PDS directory from disk can require many READ operations and hence be time-consuming. If you repeat that delay for almost every program search in every job, you have a drag on the system. So LLA gives a meaningful performance improvement for reading members from PDS-type libraries. For PDSE libraries too, but for different reasons; PDSE libraries do not have directory blocks like ordinary PDS libraries. Anyway, the price you pay for the improvement is that the in-memory copies of the directories have to be rebuilt whenever a directory is updated, that is, whenever a library member is replaced. What does LLA stand for? Originally it stood for LINKLIST LookAside, but when the concept was extended to cover other libraries besides LINKLIST the name was changed to Library LookAside.
Under TSO, CLIST and REXX execs
Under TSO, if a module is not found in any of the above places, there is a final search for a CLIST or REXX exec matching the requested name.
CLIST and REXX execs are obtained by searching ddnames SYSUEXEC, SYSUPROC, SYSEXEC, and SYSPROC (in that order, if the order has not been deliberately changed). Note that SYSUEXEC and SYSEXEC are for REXX members only, whereas SYSPROC and SYSUPROC can contain both REXX and CLIST members, with the proviso that a REXX member residing in the SYSPROC family of libraries must contain the word REXX on the first line, typically as a comment /* REXX */
These four ddnames can be changed and other ddnames can be added by use of the ALTLIB command. Also the order of search within these ddnames can be altered with the ALTLIB command (among other ways)
.SYSPROC was the original CLIST ddname. A CLIST library can also include REXX execs. The SYSEXEC ddname was added to be used just for REXX. In the spirit of ISPLUSR et al, a USER version of each was added, called SYSUPROC and SYSUEXEC.
The default REXX library ddname SYSEXEC can be changed to something other than SYSEXEC by MVS system installation parameters.
Prefixing the TSO command name with the percent sign (%) causes the system to skip directly to the CLIST and REXX part of the search, rather than looking every other place first.
To find the actual search order in effect within the CLIST and EXEC ddnames for your TSO session at any given time, use the command TSO ALTLIB DISPLAY.
That's it for program search order. It's a simplification of course. ;-)
References, Further reading
z/OS Basic Skills, Mainframe concepts, z/OS system installation and maintenance
Search order for programs
z/OS TSO/E REXX Reference, SA32-0972-00
Using SYSPROC and SYSEXEC for REXX execs
z/OS V2R2 ISPF Services Guide
Application data element search order
z/OS ISPF Services Guide, SC19-3626-00
LIBDEF—allocate application libraries
Numbered item 3 in Section “Search order the system uses for programs” in the IBM publication “z/OS MVS Initialization and Tuning Guide” at this link: https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieae100/ieae10018.htm
"The Search for the Load Module" in the z/OS MVS Assembler Services Guide
"Customizing the ISPF TSO command table (ISPTCM)" and “Customizing command tables” In the IBM manual "ISPF Planning and Customizing"