Working with Datasets Part 1 non VSAM

Working with Datasets Part 1, non VSAM

Topic Objectives Be able to: • Explain what a data set is • Describe data set naming conventions and record formats • List some access methods for managing data and programs • Explain what catalogs and VTOCs are used for • Be able to create, delete, and modify data sets

Key Terms in This Topic nblock size ncatalog ndata set nhigh level qualifier (HLQ) nlibrary nlogical record length (LRECL) nmember npartitioned data set (PDS) npartitioned data set extended (PDSE) nrecord format (RECFM) nsystem-managed storage (SMS) nvirtual storage access method (VSAM) nvolume table of contents (VTOC)

What is a data set? A data set is a collection of logically related data records stored on one disk storage volume or a set of volumes (and TAPE). § A data set can be: § • • • § a source program a library of macros a file of data records used by a processing program. You can print a data set or display it on a terminal. The logical record is the basic unit of information used by a program running on z/OS.

How data sets are named n Data set naming convention u Unique name − Maximum 44 characters u Maximum of 22 name segments: level qualifier − The first name in the left: high level qualifier (HLQ) − The last name in the right: low level qualifier (LLQ) − Level qualifiers are separated by '. ' u Each level qualifier: − − From 1 up to 8 characters The first must be alphabetical (A-Z) or special (@ # $) The 7 remaining: alphabetical, national, numeric (0 -9) or hyphen (-) Upper case only Example: MYID. JCL. FILE 2 HLQ: MYID 3 qualifiers n Member name of partitioned data set u 8 bytes long u First byte: alphabetical (A-Z) or special (@ # $) u The 7 remaining: alphabetical, special, numeric (0 -9) u

Data set Naming

What an access method is Defines the technique used to store and retrieve data. n Includes system-provided programs and utilities to define and process data sets. n Commonly used access methods include the following: n * Includes z. FS

DASD: Use and terminology § Direct Access Storage Device (DASD) is another name for a disk drive. § DASD volumes are used for storing data and executable programs. § Data sets in a z/OS system are organized on DASD volumes. • • • A disk drive contains cylinders Cylinders contain tracks Tracks contain data records.

Datasets

Using a data set To use a data set, you first allocate it. Then, access the data using macros for the access method that you have chosen. n Various ways to allocate a data set: n u u ISPF data set panel, option 3. 2 Access Method Services TSO ALLOCATE command job control language (JCL)

Allocating space on DASD volumes n How space is specified: u u n Logical records and blocks: u u n explicitly (SPACE parameter) implicitly (SMS data class) Smallest amount of data to be processed Grouped in physical records named blocks Data set extents: u Space for a disk data set is assigned in extents

What is a data set, and how is it stored

Data set record formats F record Fixed records. FB block record record Fixed blocked records. BLKSIZE = n * LRECL V record Variable records. RDW block VB BDW U record block record record Variable blocked records. BLKSIZE >= 4 + n * largest LRECL record Undefined records. No defined internal structure for access method. Record and block descriptors words are each 4 bytes long

Types of data sets n We discuss three types in this class: u Sequential, partitioned, and VSAM A sequential data set is a collection of records written and read in sequential order from beginning to end. n A partitioned data set (PDS) is a collection of sequential data sets, called members. n u u n Consists of a directory and one or more members. Also called a library. A PDSE is a partitioned data set extended.

How data is stored in a z/OS system n n n Data is stored on a direct access storage device (DASD), magnetic tape volume, or optical media. You can store and retrieve records either directly or sequentially. You use DASD volumes for storing data and executable programs, including the operating system itself, and for temporary working storage. You can use one DASD volume for many different data sets, and reallocate or reuse space on the volume. Tape and optical media is also used

General Dataset Specifications 349

Types of Non-VSAM datasets You create a directory when you create PDS dataset Means of grouping similar Artifacts (code or data) ani Different business needs not necessarily related

Partitioned Datasets = PDSE

PDS versus PDSE n PDS data sets: u n Simple and efficient way to organize related groups of sequential files. PDSE data sets: u Similar to a PDS, but advantages include: − Space reclaimed automatically when a member is deleted − Flexible size − Can be shared − Faster directory searches

Advantages of PDSE • The size of a PDSE directory is flexible and can expand to accommodate the number of members stored in it (the size of a PDS directory is fixed at allocation time). * PDSE members are indexed in the directory by member name. This eliminates the need for time-consuming sequential directory searches. * The logical requirements of the data stored in a PDSE are separated from the physical (storage) requirements of that data, which simplifies data set allocation. * PDSE automatically reuses space, without needing an IEBCOPY compress. A list of available space is kept in the directory. When a PDSE member is updated or replaced, it is written in the first available space. This is either at the end of the data set, or in a space in the middle of the data set marked for reuse. For example, by moving or deleting a PDSE member, you free space that is immediately available for the allocation of a new member. This makes PDSEs less susceptible to space-related abends than PDSs. * This space needs not be contiguous. The objective of the space reuse algorithm is not to extend the data set unnecessarily. * The number of PDSE members stored in the library can be large or small without concern for performance or space considerations. * You can open a PDSE member for output or update, without locking the entire data set. The sharing control is at member level, not the data set level. * The ability to update a member in place is possible with PDSs and PDSEs. But with PDSEs, you can extend the size of members and the integrity of the library is maintained while simultaneous changes are made to separate members within the library. * The maximum number of extents of a PDSE is 123; the PDS is limited to 16. * PDSEs are device-independent because they do not contain information that depends on location or device geometry. * PDSEs can contain program objects built by the program management binder that cannot be stored in PDSs.

Generation Datasets: GDG DSN=MYFILE. ACCOUNT. FILE(+1) DSN=MYFILE. ACCOUNT. FILE(G 00 V 0001) • All of the datasets in the group can be referred to by a common name • The Operating System is able to keep the generations in chronological order

IDCAMS Utility - LISTCAT output of GDG base MHLRES 4. TEST. GDG(+1) … MHLRES 4. TEST. GDG(+2) during job run MHLRES 4. TEST. GDG(0) …. MHLRES 4. TEST. GDG(-1) at job conclusion

GDG illustration

Allocating a Dataset in ISPF Attributes can also be entered thru JCL Note: Positional parameters JCL //DDCARD DD DSN=ROGERS. JCL. TEST, DISP=(, CATLG), // SPACE=(CYL, (5, 1, 50), , CONTIG), // DCB=(RECFM=FB, LRECL=80, BLKSIZE=27920) NOTE: DFHSMS-ACS

Catalogs and VTOCs n z/OS uses a catalog and a volume table of contents (VTOC) on each DASD volume to manage the storage and placement of data sets. n VTOC: u u Lists the data sets on a volume Lists the free space on the volume.

VTOC

Dataset Control Blocks (DSCB)

IBM Utility – IEHLIST VTOC

VTOC Index Structure ISPF option 3. 4

Max Dataset Size • Some types of datasets are limited to 65, 535 total tracks allocated on any one volume Exceptions: H F S (165. 2 GB) P D S E (123 Extents) VSAM (4 GB limit)* Special cases – Extended format data sets with multiple strips are limited to 16 volumes Tape datasets are limited to 255 volumes * Using Extendable Format = 128 TB

Volume Table of Contents First record in every VTOC Format type 4

How a catalog is used A catalog associates a data set with the volume on which the data set is located. n Locating a data set requires: n u u u n Data set name Volume name Unit (volume device type) Typical z/OS system includes a master catalog and numerous user catalogs.

Catalog Structure Basic Catalog Structure (BCS) – This is considered the “real” Catalog The BCS is a VSAM KSDS and its primary function is to point to the volumes on which the dataset is located. VSAM Volume Dataset (VVDS) – Can be considered an ‘extension’ of VTOC The VVDS is an ESDS containing information to process the dataset containing volume related information.

VSAM Volume Data Set

Integrated Catalog Structure (ICF) Volume, Security, Ownership, etc. VSAM and non-SMS Managed Datasets SYS 1. VVDS. Vvolser Basic Catalog Structure (BCS) – Static information that rarely changes VSAM Volume Data. Set (VVDS) - Additional Catalog Information

BCS – itself is a VSAM KSDS dataset Where DS resides: Tape, Disk, …other Uses Dataset Names as keys

VVDS example n n n n n n n n VSAM Utilities ------------------- LINE 0000 COL 001 080 COMMAND ===> SCROLL ===> PAGE ***************** Top of Data ***************** IDCAMS SYSTEM SERVICES TIME: 07: 41 /* IDCAMS COMMAND */ LISTCAT ENTRIES(SYS 1. VVDS. VDMPU 03) CLUSTER ------- SYS 1. VVDS. VDMPU 03 IN-CAT --- CATALOG. MASTER. MCAT DATA ------- SYS 1. VVDS. VDMPU 03 IN-CAT --- CATALOG. MASTER. MCAT IDCAMS SYSTEM SERVICES TIME: 07: 41 THE NUMBER OF ENTRIES PROCESSED WAS: AIX ----------0 ALIAS ---------0 CLUSTER --------1 DATA ---------1 } ESDS: VSAM Seq file GDG ----------0 INDEX ---------0 NONVSAM --------0 PAGESPACE -------0 PATH ---------0 SPACE ---------0 USERCATALOG ------0 TAPELIBRARY ------0 TAPEVOLUME ------0 TOTAL ---------2 THE NUMBER OF PROTECTED ENTRIES SUPPRESSED WAS 0 IDC 0001 I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0

Catalog Path Structure DEFINE ALIAS (NAME (IBMUSER) RELATE (USERCAT. IBM ) )

Where MASTER CATALOG comes from SYS 1. IPLPARM(LOADxx) Is a file stored in the Service Element used by mainframe HW to define and manage channels, control units and device paths. Note: with the proper set up, a new Hardware addresses can be deployed with Dynamic Reconfiguration. Or located in SYS 1. NUCLEUS(SYSCATxx) IEASYS xx Good documentation SYS 1. SAMPLIB(IPXLOADX) You can also do a LISTC on SYS 1 to obtain its name For automatic commands located In COMMNDxx in PARMLIB On Master Console IEA 101 A SPECIFY SYSTEM PARAMETERS REPLY 00, ’CMD=ZZ’ REPLY 00, ’SYSP=CS, CMD=‘ZZ’ REPLY 00, ’SYSP=00, CLPA, CMD=‘ZZ’

User Catalog Alias’ Good practice: different applications in different UCATs

Locating a dataset in MVS Where is Oh…its Catalogued that #$%@ GREAT ! nevermind Dataset DS z. OS 1. 7 STEPCAT JOBCAT goes away

Datasets Catalog and Uncataloged Note the ‘ // ‘ and parm statements used for Job Control Language

MVS Datasets and Unix Files

Traditional Disk Capacity (DASD) • The 3390/3380 Volume concept no longer exists • We do not know where “logical” tracks are located on disks since several changes in device geometry have occurred with introduction of new products

Large Volume (own device type)

Data management in z/OS n Data management involves all of the following tasks: u allocation, placement, monitoring, migration, backup, recall, recovery, and deletion. Storage management is done either manually or through automated processes (or through a combination or both). n In z/OS, DFSMS is used to automate storage management for data sets. n

Data Facility Subsystem Managed Storage (DFSMS)

DFSMS (System Managed Storage) Gold customer High Priority Business Partner High Priority Customer care High Priority (Business Hours) Casual customer Low priorty Data Analysis (Best can do)

DFSMS uses “rule” based management Gold customer High Priority Casual customer Low priorty Business Partner High Priority Customer care High Priority (Business Hours) Data Analysis (Best can do)

Automatic Class Selection Used to control: retention Used to control: migration Used to control: backup performance goals Allocation release availability Space

Example of a Processing ACS Routines - SYSPLEX Wide Note: ACS language is a high-level programming language. Once written you use the ACS translator to create an SCDS ACDS object form to be stored in the SMS configuration. Policy Config. Active Policy

Rule Based Policy Management to Manage Backup/Restore Automatically via Hierarchical Storage Gold customer High Priority Business Partner High Priority Data Analysis (Best can do) Tape Secondary Storage Third Storage Media

Interactive Storage Management Facility Storage Management is performed interactively via ISPF panels

Working with Datasets Part 2, VSAM

VSAM n VSAM is Virtual Storage Access Method n VSAM provides more complex functions than other disk access methods n VSAM record formats: u u Key Sequence Data Set (KSDS) Entry Sequence Data Set (ESDS) Relative Record Data Set (RRDS) Linear Data Set (LDS)

VSAM Access Method

Simple VSAM control interval Adjacent records of the same length only require 2 RDFs 3 3 3 4 bytes i. e 4096 = CI - 4 CIDF (if free space % 0) 2 RDF/Record* 80 Record length -------49 records / per CI But. . 10% freespace = 409 bytes used for inserts (4. 9 records)

Control Interval Format

VSAM KSDS CLUSTER

VSAM Index Structure

VSAM Keyed Dataset

VSAM Sequential Dataset = ESDS

VSAM - RRDS

VSAM LDS

DATA-IN-VIRTUAL (DIV) Address Space Data Space Hiper Space

Basic Parms for VSAM KSDS Dataset

VSAM Datasets with SMS (ACS) You can just provide the name

VSAM Alternate Indexes 15 36 41 Index CI unused Data 10 A-CITY TOM 12 B-CITY MIKE 15 A-CITY FRED CI 21 F-City BEN CI 23 E-CITY FRED 36 B-CITY BILL 39 A-CITY FRED 41 G-CITY TOM MIKE TOM Free Index CI unused BEN 21 BILL 36 FRED 15 23 39 MIKE 12 TOM 10 41 CI unused Free E-CITY G-CITY CI CI unused Index CI Data A-CITY 10 15 39 B-CITY 12 36 E-CITY 23 unused F-CITY 21 G-CITY 41 Data Free CI CI

When to use which dataset type Use KSDS if: – The data access is sequential, skip sequential, or direct access by a key field. – You would prefer easy programming for direct data processing. – There will be many record insertions, deletions, and logical record length varies. – You may optionally access records by an alternate index. – Complex recovery (due to index and data components) is not a problem. – You want to use data compression Use RRDS if: – The record processing is sequential, skip sequential, or direct processing. – Easy programming for direct processing is not a requirement. – The argument for accessing data in direct mode is a relative record number, not the contents of a data field (key). RRDS is suitable for the type of logical records identified by a continuous and dense pattern of numbers (such as 1, 2, 3, 4. . . ). – All records are fixed length. – There a small number of record insertions and deletions, and all the space for insertions must be pre-allocated in advance. – Performance is an issue. RRDS performance is better than KSDS, but worse than QSAM or BSAM.

When to use which dataset type (cont. ) Use ESDS if: – You are adding logical records only at the end of the data set and reading them sequentially (in the application control). – The logical record is variable length – You seldom need direct record processing by key (using AIX). – You are using a batch processing application. Use LDS if: – You want to exploit DIV. – Your application manages logical records. – Performance is an issue.

When defining a VSAM Cluster. . what are the key parameters that denotes the dataset type? KSDS ESDS LDS RRDS INDEXED NONINDEXED LINEAR NUMBERED

Summary n n n n A data set is a collection of logically related data (programs or files) Data sets are stored on disk drives (DASD) and tape. Most z/OS data processing is record-oriented. Byte stream files are not present in traditional processing, except in z/OS UNIX. z/OS records follow well-defined formats, based on record format (RECFM), logical record length (LRECL), and the maximum block size (BLKSIZE). z/OS data set names have up to 44 characters, divided by periods into qualifiers. Catalogs are used to locate data sets. VSAM is an access method that provides more complex functions than other disk access methods. z/OS libraries are known as partitioned data sets (PDS or PDSE) and contain members.