You are on page 1of 32

SORT UTILITY

SORT UTILITY

Prepared by: Date Created: Date Modified:

Anish Dosslin February 19, 2004 March 12, 2004

Anish_Dosslin Restricted

Page 1

27-Feb-2004

SORT UTILITY

Revision Number 1.0 1.0 1.1 1.2

Date February 2004 February 2004 27 February 2004 02 March 2004

Summary of Changes Original Draft Review of document Corrections after review Added: CONVERT, FTOV Section on merge added

Reviewers/Authors ID Anish_Dosslin Uma_Patil Anish_Dosslin Anish_Dosslin

Anish_Dosslin Restricted

Page 2

27-Feb-2004

SORT UTILITY

1. INTRODUCTION - SORT...........4
1.1 NECESSARY STATEMENTS.....................................................................................4 1.1.1 STEP CARD SYNTAX..........................................................................................4 1.1.2 SYSOUT STATEMENT........................................................................................4 1.1.3 SORTIN STATEMENT.........................................................................................4 1.1.4 SYSIN STATEMENT............................................................................................5 1.2 OPTIONAL STATEMENTS.........................................................................................6 1.2.1 SORTOUT STATEMENT.....................................................................................6 1.2.2 SORTOFXX STATEMENT...................................................................................6 2.1 INCLUDE......................................................................................................................7 2.1.1 INCLUDE SYNTAX..............................................................................................7 2.1.2 EXAMPLES...........................................................................................................8 2.2 OMIT............................................................................................................................14 2.3 INREC..........................................................................................................................14 2.3.1 INREC SYNTAX.................................................................................................14 2.3.2 EXAMPLES..........................................................................................................14 2.4 OUTREC......................................................................................................................16 2.5 OUTFIL........................................................................................................................16 2.5.1 OUTFIL SYNTAX...............................................................................................16 2.5.10 CONVERT..........................................................................................................20 2.5.11 FTOV..................................................................................................................20 2.6 SORT............................................................................................................................22 2.6.1 SORT CARD SYNTAX.......................................................................................22 2.7 MERGE........................................................................................................................25 2.7.1 SYNTAX...............................................................................................................25 2.7.2 EXAMPLE............................................................................................................26 2.8 SUM.............................................................................................................................28 2.8.1 SYNTAX...............................................................................................................28 2.8.4 EXAMPLES..........................................................................................................29

2 SORT CARD............................6

3 EXAMPLES............................30
3.1 EXAMPLE 1............................................................................................................30 3.2 EXAMPLE 2............................................................................................................31 3.3 EXAMPLE 3............................................................................................................32

Anish_Dosslin Restricted

Page 3

27-Feb-2004

SORT UTILITY

1. INTRODUCTION - SORT
1.1 NECESSARY STATEMENTS 1. Step Card 2. \\SYSOUT DD SYSOUT=* 3. \\SORTIN or SORTINxx 4. \\SYSIN DD 1.1.1 STEP CARD SYNTAX \\Step_name EXEC PGM=SORT The step card is where you define a STEP inside a JOB. You give the step a step name (optional) then there is a positional parameter EXEC followed by PGM/PROC. If the step is to execute a PROC then the PROC keyword is optional. The step name can be a user defined 8-character name. It can be alphanumeric with national characters, but cannot start with a number. 1.1.2 SYSOUT STATEMENT The SYSOUT statement should be specified so that the output messages from the utility can be saved there. This If the SYSOUT statement is not specified then the utility ABENDs with a user return code of 0016. This is illustrated in the figure given below.

1.1.3 SORTIN STATEMENT The SORTIN DD will contain the input file(s) that has to be sorted. The SORTIN DD can be mentioned in two different ways. Anish_Dosslin Restricted

Page 4

27-Feb-2004

SORT UTILITY

1. 2.

SORTIN SORTINxx

The first method is used when there in only one input file or if the input files can be appended one behind the other. \\SORTIN DD DSN=TMP.EXAMPLE.INPUT.FILE1,DISP=SHR The above statement takes one file as the input file. \\SORTIN DD DSN=TMP.EXAMPLE.INPUT.FILE1,DISP=SHR \\ DD DSN=TMP.EXAMPLE.INPUT.FILE2,DISP=SHR \\ DD DSN=TMP.EXAMPLE.INPUT.FILE3,DISP=SHR \\ DD DSN=TMP.EXAMPLE.INPUT.FILE4,DISP=SHR \\ DD DSN=TMP.EXAMPLE.INPUT.FILE5,DISP=SHR A number of files can be given under one SORTIN DD name by concatenating them together. As a result the five files are appended one behind the other and considered as one single file. The second method: The SORTINxx DD statements are used for MERGE fields and also used in OUTFIL control statements. A maximum of 100 DDs can be specified starting from 00 to 99, some numbers may be skipped or neglected which makes no difference. The output is not altered some of the intermediate numbers are missed or if the order of numbers is improper. The SORTINxx DD DSNs should have the same LRECL and also they should be in the proper sequence for MERGE. 1.1.4 SYSIN STATEMENT The SYSIN statement contains the actual SORT card. The sort card can be inline (coded directly) in a JCL or can be contained in a member of a PDS. 1. SORT card coded in JCL \\SYSIN DD * SORT FIELDS=COPY \* \* acts as the delimiter specifying the end of the SORT card on an inline sort card. 2. SORT card coded in a member of a PDS \\SYSIN DD DSN=TMP.EXAMPLES.PDS(CTLCARD1),DISP=SHR The a sort card coded inside a member of a PDS is called a Control card. It functions as the same way as the inline sort card. The sort card can be converted into a control card by copying the sort statements between the SYSIN statement and the delimiter and saving them in a separate member of a PDS. A delimiter should not be coded in a control card.

Anish_Dosslin Restricted

Page 5

27-Feb-2004

SORT UTILITY The JCL ABENDS with a system return code of 13 (S013 U0000) when the specified control card member is not found as mentioned in the SYSIN DD. 1.2 OPTIONAL STATEMENTS

The most commonly used optional statements are, 1. \\SORTOUT 2. \\SORTOFxx OR \\SORTOFx There are other optional JCL statements such as SORTXSUM which will be dealt at a later stage. 1.2.1 SORTOUT STATEMENT The SORTOUT DD will contain the output file which will have the sorted data. \\SORTOUT DD DSN=TMP.EXAMPLE.OUTPUT.FILE1,DISP=(MOD,CALTG,CATLG) 1.2.2 SORTOFXX STATEMENT The SORTOFx or SORTOFxx statement is used when the output files are determined by the control card (i.e. by the OUTFIL FILES statement) 1.3 METHODS OF SPECIFYING OUTPUT FILES

The output file can be referred in three different ways. 1. SORTOUT DD 2. SORTOFxx DD (with OUTFIL FILES) 3. DD_Name DD (With OUTFIL FNAMES) SORTOUT DD is the most common method. The file which is mentioned in the SORTOUT DD statement and the output is stored there. If the output DD is concatenated with more than one file, then the output records go to the first DD DSN and the other will have an empty file if they are created new or will have the same data as before the SORT if the disposition was MOD SORTOFxx DD: The output files can be mentioned in the SORTOFx or SORTOFxx DD names, where x/xx can vary from 1/01 to 99. The exact output file DD has to be explicitly mentioned in the OUTFIL FILE. This will be dealt later in the OUTFIL section. DD_Name DD (With OUTFIL FNAMES): Any DD, excluding special DD names like SYSIN, SYSOUT, SORTIN, etc, can be converted into an output sort file using OUTFIL FNAMES. The DD name is given within braces in the FNAMES. This will be dealt later in the OUTFIL section.

2 SORT CARD

Anish_Dosslin Restricted

Page 6

27-Feb-2004

SORT UTILITY The sort card will contain the condition on which the input file is to be sorted on. The sort card contains more features than just sorting, it can be used to add trailer, increase/decrease the size of the input record, insert new fields and include/omit a few records depending on certain conditions. These functions are taken care of by statements known as control statements. The control statements tell how to process the files. There are 12 control statements. Specifies an alternate collating sequence for control fields with an AQ format. Signals the end of control statements in the control card Specifies the criteria which determine whether or not records are included in the sort or the output file Reformats the input record before sort/merge process Specifies merge control fields. Specifies user exit(s). Specifies the criteria which determine whether or not records are omitted from the sort process and the output file Describes the output file(s) Reformats the output record after sort/merge process. Provides record information at various processing stages. Defines a sort or copy process and specifies sort criteria Deletes records with equal control fields and summarizes numeric fields on those records.

ALTSEQ END INCLUDE INREC MERGE MODS OMIT OUTFIL OUTREC RECORD SORT SUM

2.1 INCLUDE
The INCLUDE condition to include only the required records. This is achieved by comparing a field with a constant or another field in the same record. The INCLUDE and OMIT condition cannot appear in the same SORT card. 2.1.1 INCLUDE SYNTAX INCLUDE COND=NONE ALL operator,Condition3) Logical Operator AND, & OR, | ConditionX Both AND and & can be used interchangeably as the logical AND Both OR and | can be used interchangeably as the logical OR (Condition1,Logical operator,Condition2,Logical

Anish_Dosslin Restricted

Page 7

27-Feb-2004

SORT UTILITY There can be any number of conditions. The conditions can be separated by braces to improve readability. The syntax of the condition where the comparison is done with a constant contains 5 parts. Starting Byte position, No. of bytes, Format, Operator, constant value Example: INCLUDE COND=(5,5,PD,EQ,11111) INCLUDE COND=(1,3,CH,EQ,CAL,&,5,5,PD,LT,50000) Starting Byte position: This is the relative byte position in the record. The first byte is considered as 1 and so forth. No. of Bytes: The number of bytes to be considered from the starting byte position for the comparison. The starting byte specified in the starting-byte-position field is also considered. Format: The format is specifies the type of data contained in the bytes that were selected for the process. The most commonly used formats are, 1. BI Binary (1 to 4092 bytes) 2. PD Packed decimal (1 to 256 bytes) 3. ZD Zoned decimal (1 to 256 bytes) The first 4 bits (nibble) of the rightmost byte is the sign nibble. XF/C/E/A) denotes positive, XD/B denotes negative 4. CH Character 5. SS Substring. It searches for the string mentioned in the constant field and tries to find a match of it an anywhere inside the bytes specified for processing then the record is considered for processing Operator: The operator mentioned below are the most commonly used. 1. EQ Equal 2. NE Not equal 3. GT Greater than 4. GE Greater than or equal to 5. LT Less than 6. LE Less than or equal to Constant Value: The constant value is compared with the bytes mentioned of the processing. 1 COBELIX Character constant 2 X00122C constant (represents 122 in PIC S9(3) COMP-3) 3 X'40' constant (represents a SPACE) 3 122 Numeric constant 2.1.2 EXAMPLES Sample Control Cards and their explanations: Anish_Dosslin Restricted CH CH

CH PD, BI, ZD

Page 8

27-Feb-2004

SORT UTILITY

A SAMPLE COPY BOOK: CPYSMPL1

WS-SAMPLE-CPYBOOK-1. 05 WS-NAME 05 WS-EMPLOYEE-NUM 05 WS-AGE 05 WS-DATE-OF-BIRTH 05 WS-DESIGNATION 05 FILLER

PIC PIC PIC PIC PIC PIC

X(15). 9(06). S9(02) USAGE COMP. S9(09) USAGE COMP-3. X(30). X(02).

Figure 1.1 INCLUDE COND=(1,15,CH,EQ,COBELIX) This will include the records that have the characters OBELIX in the first fifteen bytes of the record, starting only from the first byte.

Anish_Dosslin Restricted

Page 9

27-Feb-2004

SORT UTILITY

INCLUDE COND=(22,2,BI,NE,25) The starting byte is 22 and number of bytes, 2 corresponds to the starting byte position of WS-AGE field. As format specified is binary (BI) and as a result the, only records that are not equal to 25 are included.

INCLUDE COND=(16,6,CH,GT,C025000) This condition includes the records that has a value greater than 25000 in the specified byte locations. In the specified byte location the format contained is of 9(08). As PIC 9(08) is interchangeable with PIC X(08), the data is considered as character. The same can be achieved by INCLUDE COND=(16,6,ZD,GT,025000)

Anish_Dosslin Restricted

Page 10

27-Feb-2004

SORT UTILITY

INCLUDE COND=(24,5,PD,EQ,19810626) INCLUDE COND=(24,5,CH,EQ,X019810626C) Both the conditions, include the records that have the 19810626 in PD format in the specified byte locations. The second condition represents the comparison field in the hexadecimal format and the Format is given as CH and not as PD.

The hexadecimal format comes useful when you have a compound key. For example, you have a key as follows. 1 WS-KEY.

Anish_Dosslin Restricted

Page 11

27-Feb-2004

SORT UTILITY 05 WS-YEAR 05 WS-DEPARTMENT 05 WS-ROLL-NUMBER PIC S9(05) COMP-3. PIC X(03). PIC S9(03) COMP-3.

Lets say WS-YEAR = 1998, WS-DEPARTMENT = CSE and WS-ROLL-NUMBER = 2. This can be represented in hexadecimal format as X01998CC3E2C5002C. The hexadecimal equivalent in 1998 (S9(05) COMP-3/PD) is X01998C, where C represents positive. This occupies 3 bytes and hence 6 hexadecimal digits. The hexadecimal equivalent in CSE is XC3E2C5, where XC3 represents C, XE2 represents S and XC5 represents E. The hexadecimal equivalent in 2 (S9(03) COMP-3/PD) is X002C, where C denotes positive. This occupies 3 bytes and hence 4 hexadecimal digits. INCLUDE COND=(1,1,CH,LE,CG,AND,22,2,BI,LE,25) INCLUDE COND=(1,1,CH,LE,CG,&,22,2,BI,LE,25) This sort card includes the names those have the starting letter as a character less than G (i.e. A, B, C, D, E, F) and those corresponding age should be less than or equal to 25. The logical operator is AND which can also be represented by &.

INCLUDE COND=(1,15,SS,EQ,CNIX,OR,29,30,SS,EQ,CSOFTWARE) INCLUDE COND=(1,15,SS,EQ,CNIX,|,29,30,SS,EQ,CSOFTWARE) This include the records that have the string NIX in any of the first fifteen bytes or the string SOFTWARE is present starting from any position in the thirty bytes mentioned in the second condition. The two conditions are linked by an OR. The OR can also be represented by |.

Anish_Dosslin Restricted

Page 12

27-Feb-2004

SORT UTILITY

The syntax of the condition where the comparison is done with another field in the same record constant contains 5 parts. Starting position of 1st field, No. of bytes, operator, Starting position of 2nd field, No of bytes Example: INCLUDE COND=(2,1,NE,30,1),FORMAT=CH This condition will check the second byte (2,1,NE,30,1) with the thirtieth byte (2,1,NE,30,1) and will include the records which have these two fields are not equal; both the fields be considered in the character format.

Anish_Dosslin Restricted

Page 13

27-Feb-2004

SORT UTILITY If the any one of the fields under consideration has more bytes than the other, then the smaller field is padded either with ZERO in case of ZD, PD and BI or with SPACE incase of CH. Let us consider COND=(2,1,NE,30,5),FORMAT=CH In this case, a one byte character is being compared with a five byte character field. As one of the fields is smaller, the smaller field is taken and padded with SPACE/ZERO so that both the fields occupy the same number of bytes.

2.2 OMIT
The OMIT COND operates in the same way as the INCLUDE COND, but the difference is that the records that pass the condition are omitted in this case.

2.3 INREC
INREC is used to reformat the records before the sort process actually happens. This statement can be used to add or delete fields, reformat one type of data to another, etc. Note: This INREC process happens before the INCLUDE or OMIT conditions or before the sort. 2.3.1 INREC SYNTAX INREC FIELDS=(Starting position in reformatted record: Starting position in input record, Number of bytes to be copied, Old field format [optional], New field format [optional] ) Note: The condition can be repeated multiple times with different criteria Another representation: INREC FIELDS=(Starting position in reformatted record: n{X, Xhh..,C ,Z}) n Positive integer X Inserts SPACE (repeated n times) Xhh.. Inserts a hexadecimal value (repeated n times) C Inserts the string mentioned inside the quotes (repeated n times) 2.3.2 EXAMPLES A SAMPLE COPY BOOK: CPYSMPL2 01 WS-SMPL-CPYBOOK-2. 05 WS-EMPLOYEE-NUM 05 WS-NAME 05 WS-AGE 05 WS-DESIGNATION Anish_Dosslin Restricted PIC PIC PIC PIC 9(06). X(15). S9(03) USAGE COMP-3. X(12).

Page 14

27-Feb-2004

SORT UTILITY

Example to convert the copy book CPYSMPL1 to CPYCMPL2: INREC FIELDS=(1:16,6, 7:1,15, X'021D', 24:29,8, 32:X, 33:3C'T') This statement copies 6 bytes (1:16,6,) from the 16th byte from the input record (1:16,6,) to the 1st byte (1:16,6,) in the output record. Similarly it copies 15 bytes (,7:1,15,) from the (,7:1,15,) 1st byte in the input record to the 15 byte positions starting from the 7th position (,7:1,15,) in the output record. The third condition is a simple insert of the hexadecimal value 021D. As the corresponding field is a COMP-S field, the value inserted into all the records in the output file is -21 as the D in 021D stands for negative value representation. The fourth condition copies eight bytes from the 19 th byte in the input file to eight bytes starting from the 24th byte in the output file. The fifth condition inserts a SPACE in the 32nd byte. The six condition inserts 3 characters T (33:3C'T') in the output record starting from the 33rd position (33:3C'T'). If the size of the output record is not the same as the input record, then the LRECL parameter of the output file in the JCL should be made equal to the output record. Sample JCL //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL04 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1, // DISP=SHR //SORTOUT DD DSN=DDST.DOSSLA2.FILES.BOK.INREC, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=35,RECFM=FB,DSORG=PS) //SYSIN DD * INREC FIELDS=(1:16,6, 7:1,15, X'021D', 24:29,8, 32:X, 33:3C'T') SORT FIELDS=COPY /*

Anish_Dosslin Restricted

Page 15

27-Feb-2004

SORT UTILITY

2.4 OUTREC
The OUTREC works in the same way as the INREC control statement. It is used to accomplish these tasks:

Delete or repeat portions of the input record in the same record. Insert character strings between data fields in a record. Insert binary zeros. Create a sequence number field. Convert numeric data to printable format or to another numeric data format. Perform arithmetic operations (multiply, divide, add, subtract) and minimum and maximum functions with numeric fields and constants. Convert data to printable hexadecimal format. Select, realign, and reorder data fields. Convert a variable length record input file to a fixed length record output file.

2.5 OUTFIL
The OUTFIL control statement is primarily used to write the output into multiple output files. It can also be used as a sort writer to write headers, trailers, etc 2.5.1 OUTFIL SYNTAX OUTFIL FILES FILES(XX) FNAMES=(DD Name1, DD Name2 ) ALL NONE INCLUDE OMIT STARTREC=n ENDREC=n SAVE SPLIT OUTREC=(field1, field2 ... ) CONVERT FTOV HEADER1=(field1, field2 ... ) HEADER2=(field1, field2 ... ) TRAILER1=(field1, field2 ... ) TRAILER2=(field1, field2 ... ) SECTIONS=(field1, field2 ... ) LINES={n | ANSI | (ANSI,n) NODETAIL 2.5.2 FILES(XX) (Optional) The files sub parameter can be used if there are SORTOFx or SORTOFxx statements and the XX in the FILES(XX) will represent the number that is contained in the FILES(XX) control statement. Anish_Dosslin Restricted

Page 16

27-Feb-2004

SORT UTILITY

2.5.3 FNAMES (Optional) Mentions the output DD names that will be the output files for this FNAMES control. The DD name is of 8 characters. For example FNAMES(DDNAME1,DDNAME2,DDNAME2) connects the OUTFIL control statement with these output files. 2.5.4 INCLUDE/OMIT (Optional) The INCLUDE/OMIT conditions in OUTFIL has the same syntax as the INCLUDE COND/OMIT COND control statements. The only difference is here, COND is not mentioned. The condition will look like OUTFIL FNAMES=(DDNAME1) INCLUDE=(1,5,CH,EQ,COBELIX) 2.5.5 STARTREC=n (Optional) The STARTREC specifies the nth record from which the processing is to be done. All records found before n will be neglected. 2.5.6 ENDREC=n (Optional) The ENDREC specifies the nth record after which no processing is done and the n+ records are neglected. ENDREC does not mean that the processing is done for n records. It means that the processing stops after the nth record. //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL05 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1,DISP=SHR //OPFILE DD DSN=DDST.DOSSLA2.FILES.BOK.OUTREC, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=COPY OUTFIL FNAMES=OPFILE, STARTREC=3, ENDREC=5 /* The output should have only 3 records, i.e. starting from the 3rd records (STARTREC=3 and ENDREC=5).The input file is given in figure 1.1. The output is shown in the picture below.

Anish_Dosslin Restricted

Page 17

27-Feb-2004

SORT UTILITY

2.5.7 SAVE (Optional) The save parameter is used to include the records that were excluded in other OUTFIL groups. If SAVE is specified for an OUTFIL group it gets the records that were excluded by the OUTFIL groups that did not have the SAVE option. Let us assume that five records were excluded in one of the OUTFIL groups. Then there excluded records will be considered as input records for the OUTFIL group that has SAVE option specified in it. 2.5.8 INREC/OUTREC (Optional) The INREC/OUTREC in OUTFIL control statement is similar to the INREC FIELDS/OUTREC FIELDS control statement. The difference being the FIELDS is not mentioned in the JCL for OUTFIL. 2.5.9 SPLIT - The SPLIT parameter causes the output records to be split among the files in the OUTFIL. The records are distributed in rotation to the files specified in the FILES parameter. The files can be referred to by the xx value in their SORTOFx or SORTOFxx DD name. Example: //TDDSE6BK // //BOKJCL06 //SYSOUT //SORTIN //SORTOF01 // // // //SORTOF02 // Anish_Dosslin Restricted JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, MSGCLASS=X,NOTIFY=&SYSUID EXEC PGM=SORT DD SYSOUT=* DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1,DISP=SHR DD DSN=DDST.DOSSLA2.FILES.BOK.SPLIT01, DISP=(NEW,CATLG,DELETE), SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, DCB=(LRECL=60,RECFM=FB,DSORG=PS) DD DSN=DDST.DOSSLA2.FILES.BOK.SPLIT02, DISP=(NEW,CATLG,DELETE),

Page 18

27-Feb-2004

SORT UTILITY // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SORTOF03 DD DSN=DDST.DOSSLA2.FILES.BOK.SPLIT03, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=COPY OUTFIL FILES=(01,02,03), SPLIT, STARTREC=2, ENDREC=11 /* As per the above given JCL, the records are distributed to all the three files in rotation. In other words, the first record that is processed is given to the first file, the second to the second file, the third to the third file and the fourth to the first file and so on.

DD Name: SORTOF01 DSN: DDST.DOSSLA2.FILES.BOK.SPLIT01 This file contains the 2nd, 5th, 8th and 11th record. As the SPLIT statement is processed along the STARTREC and ENDREC statements, the first record is not processed at all. And the last record to be processed is the 11 th record in the input file.

DD Name: SORTOF02 Anish_Dosslin Restricted

Page 19

27-Feb-2004

SORT UTILITY DSN: DDST.DOSSLA2.FILES.BOK.SPLIT02

DD Name: SORTOF03 DSN: DDST.DOSSLA2.FILES.BOK.SPLIT03 Note: Screening of records can be done using the INCLUE/OMIT conditions and also, the records can be reformatted with the OUTREC parameter. Report writing parameters like HEADER, TRAILERn can not be used with the SPILT option in OUTFIL. Note: For the same example if SPLIT is not specified, all the output datasets will have the same records (all the records). 2.5.10CONVERT CONVERT can be used to convert a variable blocked dataset to a fixed blocked dataset. In this case the DCB of the output dataset has to have record length which is four bytes lesser than the variable blocked dataset. The variable blocked file contains control information like length, etc in the first four bytes of every record. CONVERT can be used only with OUTREC. In the OUTREC, we need to mentions the conditions so that the 4 control bytes in the VB dataset are neglected. 2.5.11FTOV FTOV converts a fixed block dataset to a variable block dataset. In this case the DCB of the output dataset has to have record length which is four bytes more than the variable blocked dataset. Example for CONVERT and FTOV: The following example illustrates the usage of COVERT and FTOV in the same JCL but in two different steps. The first step copies the records from the input FB dataset which have the age field less than 30 and the second step copies the records whose age field is equal to 22. The input file to the second step is the output file of the first step. //TDDSE6BK // //BOKJCL8A //SYSOUT //SORTIN Anish_Dosslin Restricted JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, MSGCLASS=X,NOTIFY=&SYSUID EXEC PGM=SORT DD SYSOUT=* DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1,DISP=SHR Page 20 27-Feb-2004

SORT UTILITY //SORTOF1 DD DSN=DDST.DOSSLA2.FILES.BOK.CONVERT1, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE), // DCB=(LRECL=64,RECFM=VB,DSORG=PS) //SYSIN DD * SORT FIELDS=COPY OUTFIL FILES=1, FTOV, INCLUDE=(22,2,BI,LT,30) /* //BOKJCL8B EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CONVERT1,DISP=SHR //SORTOF1 DD DSN=DDST.DOSSLA2.FILES.BOK.CONVERT2, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE), // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=COPY OUTFIL FILES=1, INCLUDE=(26,2,BI,EQ,22), OUTREC=(1:5,60), CONVERT /* OUTPUT OF FIRST STEP: STEP BOKJCL8A FTOV

OUTPUT OF SECOND STEP: STEP BOKJCL8B CONVERT

Anish_Dosslin Restricted

Page 21

27-Feb-2004

SORT UTILITY

2.6 SORT
The sort card specifies the exact sort criteria. The sort can be used to sort a field (a group of consecutive byte locations) either in the ascending or descending order. It can also be used to copy all the records without any change. 2.6.1 SORT CARD SYNTAX SORT FIELDS=COPY =(Starting Position, length, format, order of sort, ) EQUALS/NOEQUALS SKIPREC=n STOPAFT=n Starting position Length Format This is the starting position of the sort field - this is the number of bytes from the starting position that should be considered for sorting. - CH: Character PD: Packed decimal (COMP-3) ZD: Zoned decimal BI: Binary (COMP) - Ascending / Descending (A/D)

Order

The same can be repeated for different fields. If there are multiple sort criteria, the records will be sorted first on the first criteria, then the second criteria and so on. EXAMPLE SORT FIELDS=(22,2,BI,D,16,6,ZD,A) Note: The second SORT criteria refers to 6 bytes because of the copybook layout. The employee number can be up to 6 digits. Let us assume that there are 10 records in the file with data as given in the table below. Only the fields being considered are given below. Before sort Employee Num (16,6,ZD,A) 44444 14141 12001 15178 45365 12130 20045 25155 54653 After Sort Employee Num (16,6,ZD,A) 12001 44444 12130 15178 54653 45365 20045 14141 25155

Age (22,2,BI,D) 38 22 45 28 26 33 25 22 28 Anish_Dosslin Restricted

Age (22,2,BI,D) 45 38 33 28 28 26 25 22 22

Page 22

27-Feb-2004

SORT UTILITY

22

43522

22

43522

The sort is done so that first all the records that satisfy the first sort criteria are grouped together. And the second sort is done within those grouped records. In this example, the records with age field as 22 are grouped and then they are sorted in the ascending order of the employee number. 2.6.2 EQUALS/NOEQUALS (Optional) The EQUALS option instructs SYNCSORT to write the records with the same sort key in the same order as in the input file. If the option is NOEQUALS the records may or may not be written in the same order as the input file. NOEQUALS is the default option. Example: SORT FIELDS=(22,2,BI,D,16,6,ZD,A),EQUALS Let us consider this sort card for the data given below. I have added two more columns to the input, name and the designation. I have added two more records when compared to the previous sample input data. Both are records for the same key, Age: 22 and Employee Num: 14141. Before sort Employee Num (16,6,ZD,A) 44444 14141 12001 15178 45365 12130 14141 20045 25155 54653 14141 43522

Name 1,15 (Not Key) Getafix Obelix Unhygenix Asterix Quickfix Vitalstatistix Obelix Cacafonix Dogmatix Fulliautomatix Obelix Geriatrix

Age (22,2,BI,D) 38 22 45 28 26 33 22 25 22 28 22 22

Designation 29,30 (Not Key) SPM PL BM PM PA DM CCM PA SE PM PM SE

The input file has 12 records and three of the have the same sort key. In such a case as the EQUALS parameter is specified, the records appear in the same order as the input file. This may not be the same if the option is not specified. After Sort Employee Num (16,6,ZD,A)

Name 1,15 Key) Anish_Dosslin Restricted

Age (Not (22,2,BI,D)

Designation 29,30 (Not Key)

Page 23

27-Feb-2004

SORT UTILITY Unhygenix Getafix Vitalstatistix Asterix Fulliautomatix Quickfix Cacafonix Obelix Obelix Obelix Dogmatix Geriatrix 45 38 33 28 28 26 25 22 22 22 22 22 12001 44444 12130 15178 54653 45365 20045 14141 14141 14141 25155 43522 BM SPM DM PM PM PA PA PL CCM PM SE SE

2.6.3 SKIPREC=n (Optional) If SKIPREC is specified, the sort process will be only after n records are skipped from the input file. Is INCLUDE/OMIT parameter is specified, then these n records are omitted before the INCLUDE/OMIT processing takes place. 2.6.4 STOPAFT=n (Optional) The STOPAFT parameter specifies the number of records that should be considered for processing. This will be the first n records. If SKIPREC and/or INCLUDE/OMIT is specified, this will be the first n records after those processing is completed. Example 1: SORT FIELDS=(22,2,BI,D,16,6,ZD,A), SKIPREC=4 Intermediate step (before SORT and after SKIPREC) Employee Age Num (22,2,BI,D) 26 33 22 25 22 28 22 22 Example 2: SORT FIELDS=(22,2,BI,D,16,6,ZD,A), STOPAFT=5 Intermediate step (before sort) Age Employee Num After complete process Age Employee Num (16,6,ZD,A) 45365 12130 14141 20045 25155 54653 14141 43522

After complete process Age (22,2,BI,D) 33 28 26 25 22 22 22 22 Employee Num (16,6,ZD,A) 12130 54653 45365 20045 14141 14141 25155 43522

Anish_Dosslin Restricted

Page 24

27-Feb-2004

SORT UTILITY

(22,2,BI,D) 38 22 45 28 26

(16,6,ZD,A) 44444 14141 12001 15178 45365

(22,2,BI,D) 45 38 28 26 22

(16,6,ZD,A) 12001 44444 15178 45365 14141

Example 3: SORT FIELDS=(22,2,BI,D,16,6,ZD,A), SKIPREC=1, STOPAFT=7 Intermediate sort) Age (22,2,BI,D) 22 45 28 26 33 22 25 step (before After complete process Age Employee Num (22,2,BI,D) (16,6,ZD,A) 45 12001 33 12130 28 15178 26 45365 25 20045 22 14141 22 14141

Employee Num (16,6,ZD,A) 14141 12001 15178 45365 12130 14141 20045

2.7 MERGE
The MERGE control statement functions almost in the same way as SORT. It can be used to merge input files and also copy into one output files from multiple input files. 2.7.1 SYNTAX MERGE FIELDS=(Starting position, length, format, ) (Condition 1, operator, Condition 2 ),FORMAT=format COPY STOPAFT SKIPREC EQUALS/NOEQUALS All the parameters work in the same way as they do for SORT. But there are some factors in applying them.

1. COPY will work for multiple input files only is they or concatenated only with 2.

3.

the SORTIN statement. SORTINx/SORTINxx statements are not taken into account if SORT FIELDS=COPY is specified. For MERGE other than COPY, the input datasets should have already been sorted in the same order in which the MERGE sort is going to be performed. Else the JCL with ABEND with a MAXCC 16. It will also display a message in SYSOUT (OUT OF SEQUENCE SORTINxx) SKIPREC/STOPAFT can be mentioned only for MERGE FIELDS=COPY

Anish_Dosslin Restricted

Page 25

27-Feb-2004

SORT UTILITY 2.7.2 EXAMPLE The following example uses the copybook 01 WS-SAMPLE-CPYBOOK-4. 05 WS-ROLL-NO 05 WS-FRST-YR-MRKS 05 WS-SCND-YR-MRKS 05 WS-THRD-YR-MRKS 05 WS-FRTH-YR-MRKS CPYSMPL4. PIC PIC PIC PIC PIC S9(05) S9(04) S9(04) S9(04) S9(04) COMP-3. COMP. COMP. COMP. COMP.

SORTIN01

SORTIN02

SORTIN03

Anish_Dosslin Restricted

Page 26

27-Feb-2004

SORT UTILITY

SORTIN04 The JCL looks like the following, //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL09 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN01 DD DSN=DDST.DOSSLA2.FILES.BOK.MERGE.SORTIN01, // DISP=(SHR,DELETE,DELETE) //SORTIN02 DD DSN=DDST.DOSSLA2.FILES.BOK.MERGE.SORTIN02, // DISP=(SHR,DELETE,DELETE) //SORTIN03 DD DSN=DDST.DOSSLA2.FILES.BOK.MERGE.SORTIN03, // DISP=(SHR,DELETE,DELETE) //SORTIN04 DD DSN=DDST.DOSSLA2.FILES.BOK.MERGE.SORTIN04, // DISP=(SHR,DELETE,DELETE) //SORTOUT DD DSN=DDST.DOSSLA2.FILES.BOK.MERGE.SORTOUT, // DISP=(NEW,CATLG,DELETE), // SPACE=(TRK,(1,1),RLSE), // DCB=(LRECL=11,DSORG=PS,RECFM=FB) //SYSIN DD * MERGE FIELDS=(1,3,PD,A) /* The output file is the merged and resorted file.

Anish_Dosslin Restricted

Page 27

27-Feb-2004

SORT UTILITY

The merge sort will be successful only if the input datasets are presorted on the order in which it is going to be merge-sorted.

2.8 SUM
The SUM control statement deletes the duplicate records that have the same sort key when before they write it into the output file. In other words, the output file will have only one record corresponding to one sort key. The records with the same key are processed pair by pair. If the SUM fields are numeric fields, then the final sum can be written into one record provided there is no overflow. The records omitted from the INPUT file can be optionally written to another file. The SUM control cannot be used in conjunction with SORT FIELDS=COPY. 2.8.1 SYNTAX SUM FIELDS=(Starting position, length, format, ) (Starting position, length, ),FORMAT=X NONE XSUM 2.8.2 NONE (Optional) When NONE is specified, the number of records with the same sort key is reduced to one without summarizing the numeric fields. 2.8.3 XSUM (Optional) When XSUM is specified, the omitted records can be written to a dataset specified in a SORTXSUM DD statement. The BLKSIZE DCB of SORTXSUM is determined by the system and will be appropriate. Points to be noted: If EQUALS is specified the first record will be retained. If NOEQUALS or nothing is specified, then the record retained will be determined at random. The EQUALS parameter can be mentioned in SORT statement or can be passed as a PARM. The sort key cannot be SUMMED. So the byte locations of the sort key cannot be a part of the SUM key. The sum fields should not overlap each other. If arithmetic overflow occurs, then records are not summarized and both the records are retained. Further if the same key is processed and records are found, the records will be summarized provided overflow of the summed fields do not occur. In order to prevent this, the field size (and hence the record size) may be varied using the INREC statement.

Anish_Dosslin Restricted

Page 28

27-Feb-2004

SORT UTILITY 2.8.4 EXAMPLES A SAMPLE COPY BOOK: CPYSMPL3

WS-SAMPLE-CPYBOOK-3. 05 WS-NAME 05 WS-EMPLOYEE-NUM 05 WS-DATE-OF-BIRTH 05 WS-AGE 05 WS-DESIGNATION 05 FILLER

PIC PIC PIC PIC PIC PIC

X(15). S9(07) USAGE COMP-3. S9(09) USAGE COMP-3. S9(02) USAGE COMP. X(30). X(04).

1.

Example: Complete JCL

//TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL07 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL3,DISP=SHR //SORTOUT DD DSN=DDST.DOSSLA2.FILES.BOK.SUM, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=(16,4,PD,A,20,5,PD,A) SUM FIELDS=(25,2,BI) /*

Anish_Dosslin Restricted

Page 29

27-Feb-2004

SORT UTILITY

2.

SUM FIELDS=(16,4,20,5),FORMAT=PD

When the sort key is equal for two records, it adds the numeric data in the two fields in the two records respectively and makes them into one record. In other words, the data contained in four bytes starting from 16th byte in both the records are summed up together and contained in the five bytes starting from the 20 th byte are summed together. As both the fields have the same format, the FORMAT parameter is taken out and not specified for each field. 3. SUM FIELDS=(16,4,BI),XSUM

When the sort key is the same the data in the specified byte locations are summed together and one record with the summed data is written into the output file. As the XSUM parameter is mentioned, the omitted record is written to the dataset mentioned in the SORTXSUM DD.

3 EXAMPLES
3.1 EXAMPLE 1 A sort JCL to sort the input file in term of the employee number and then split the records into two files so that the employees over 30 years old are in one file and the others in another file. The file layout is based on the sample copybook CPYSMPL1. //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL01 EXEC PGM=SORT //SYSOUT DD SYSOUT=* Anish_Dosslin Restricted

Page 30

27-Feb-2004

SORT UTILITY //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1, // DISP=SHR //OUTGE30 DD DSN=DDST.DOSSLA2.FILES.BOK.OUTGE30, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //OUTLT30 DD DSN=DDST.DOSSLA2.FILES.BOK.OUTLT30, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=(16,6,ZD,A) OUTFIL FNAMES=OUTGE30, INCLUDE=(22,2,BI,GE,30) OUTFIL FNAMES=OUTLT30, INCLUDE=(22,2,BI,LT,30) /* This JCL uses OUFIL control to select the output file. Hence the SORTOUT DD is not necessary. The file is sorted in the ascending order of employee number and then depending on the condition, the records are distributed among the two output files. 3.2 EXAMPLE 2 A sort JCL to sort the input file in term of the employee number and then split the records into three files in an equal rotation basis (using SPLIT). The input file layout is also modified from CPYSMPL1 to CPYSMPL2 using OUTREC control within the OUTFIL. //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //BOKJCL02 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.CPYSMPL1, // DISP=SHR //SORTOF1 DD DSN=DDST.DOSSLA2.FILES.BOK.OUT1, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=35,RECFM=FB,DSORG=PS) //SORTOF2 DD DSN=DDST.DOSSLA2.FILES.BOK.OUT2, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=35,RECFM=FB,DSORG=PS) //SORTOF3 DD DSN=DDST.DOSSLA2.FILES.BOK.OUT3, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=35,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=COPY OUTFIL FILES=(1,2,3), SPLIT, OUTREC=(1:16,6,7:15C'A',X'025D',24:29,12) /*

Anish_Dosslin Restricted

Page 31

27-Feb-2004

SORT UTILITY The JCL contains one input file. As the sub parameter used to specify the output files is OUTFIL FILES we have to use the SORTOFx or SORTOFxx DD statements to specify the output file names. The LRECL of the output files DCB is 35 depending on the output copybook structure. The OUTREC sub-control moves the six bytes from the 16th position in the input file to the 6 positions in the output file 1:16,6 Then from the 7th byte in the output copybook, 15 characters A are inserted using 7:15CA. Tip: 15 Spaces can be inserted using 7:15X (in general representation Starting position:nX). Then a hexadecimal value X025D is inserted which represents -25 in COMP-3 format. Then, 12 bytes from the 29 th position in the input file are copied to the 24 th position in the output file. 3.3 EXAMPLE 3 A sort JCL to sort the input file in the descending order of the employee number and then SUM the ages found in file. The file layout is based on the copybook CPYSMPL1. //TDDSE6BK JOB (0668,M3),'ANISH DOSSLIN',CLASS=A, // MSGCLASS=X,NOTIFY=&SYSUID //DELFILES EXEC PGM=IEFBR14 //DD1 DD DSN=DDST.DOSSLA2.FILES.BOK.SORTOUT, // DISP=(MOD,DELETE,DELETE) // SPACE=(CYL,(10,10),RLSE), // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //BOKJCL03 EXEC PGM=SORT //SYSOUT DD SYSOUT=* //SORTIN DD DSN=DDST.DOSSLA2.FILES.BOK.DUPLICTE, // DISP=SHR //SORTOUT DD DSN=DDST.DOSSLA2.FILES.BOK.SORTOUT, // DISP=(NEW,CATLG,DELETE), // SPACE=(CYL,(10,10),RLSE),UNIT=SYSDA, // DCB=(LRECL=60,RECFM=FB,DSORG=PS) //SYSIN DD * SORT FIELDS=(16,6,ZD,D) SUM FIELDS=(22,2,BI) /* The sort is done on the employee number found from the 16 th position and is 6 bytes long. Then the age field (2 bytes from the 22 nd position) is summed up. The SUMming will not work if the sort criteria are not specified because the adjacent records might not have the same key. In order for the SUM condition to work all the sorted similar key fields should be adjacent to each other. The STEP given in dark gray can be used to delete the output file using a step, so that we do not have to manually delete the output file (which was created in the previous run) when we want to run the job again. If there are more than one output files then add a DD corresponding to the other datasets that have to be deleted. It is better to code the DCB and SPACE parameters so that the step will function properly whether the dataset exists or not. Anish_Dosslin Restricted

Page 32

27-Feb-2004