The RA-FTP Solution in combination with a Mainframe based FTP Server
The graphical user interface of the RA-FTP Solution (further referred to as the GUI) does not fully support the specifics of certain Mainframe (MVS, z/OS, OS390, AS400 and Siemens BS2000) file systems. This does not hinder users deploying the RA-FTP Solution for transferring files from or to those mainframe systems. But it requires some knowledge of the file system and its specifics. By specifying correct file names in the various fields used for job definition, a user can successfully create jobs to do the task at hand. This article will give an explanation of the specifics of this file system and how certain ftp actions act specifically within this file system.
Basic mainframe file system knowledge
This article is not intended to give a full introduction to the mainframe file system. But it will point out some theories that might be necessary for a user of the RA-FTP solution to be aware of. There are various sources of information available on the internet that can be used to get deeper knowledge whenever required. In many cases the user might already have this knowledge or has access to mainframe operators to get this information.
Actually there are two file systems available on mainframe systems
- The POSIX-style zFS former called HFS.
This is a “relatively new” file system that is still not readily available. It is POSIX-like which means similar to UNIX and therefore no problem at all. It can be accessed through a mainframe FTP-Server and as soon as the user enters its root path in this file system they should have no difficulties with our RA-FTP Solution GUI.
There still can be an initial problem directly after the connection is established. Normally there is only one FTP-Server running on the mainframe and if it is configured to put the user in the regular MVS file system after login a user might consider some problems within the GUI. They will need to enter the zFS (HFS) path first manually to access the zFS.
This article will not go deeper into zFS but will focus on an older and much more frequently used MVS file system. Note that all further information such as having no directories does not refer to zFS.
- The regular MVS, BS2000 and AS400 file systems.
This is most frequently used on mainframe systems. There is, in many cases, a large amount of very old legacy programs running that need to be fed with data and those applications do not have access to file systems other than the regular MVS file system. So a common application for our RA-FTP solution is to store data on a mainframe system for further processing by this legacy applications or to have processed data returned back to other systems by using RA-FTP to get the data onto a Windows or UNIX system.
The MVS file systemThere are no directories!
Most users know about Windows or UNIX file systems. One important feature of these file systems are directories. This concept does not exists in the base file system of a mainframe system.Dataset
A dataset can be seen as a file. Datasets have various parameters associated with them. Basically these parameters define the format and size. Applications on mainframes strongly relate to the format of their input data. Prominent examples of this formats are:
- Fixed block - each line has a fixed length
- Variable block - each line has variable length
- Partitioned - contains “members”
- Library – used for executables
So, in many cases, it is important to create a file in the proper format. The mainframe ftp server creates per default files in variable block format and by using the “Make Directory” command a partitioned dataset. The format of the created files can be changed by specific “Quote Site” commands. It is important to know that this is possible. For details please refer to the documentation of the mainframe ftp. It can be found on the internet or in the ftp server manual.
A valid dataset name consists of qualifiers separated by dots (.) and must not exceed 44 bytes in total length. Each qualifier must start with alphabetical character (not a number).
TEST.RAFTP.INPUT.X1 is a valid data set name. It consist of the qualifiers TEST, RAFTP, INPUT and X1. Qualifiers are important on a mainframe system as they can be used to structure data. So datasets can be listed by using * as wildcard character. So listing datasets with the pattern TEST.RAFTP.* will list all datasets starting with the two qualifier TEST and RAFTP. Note that this can simulate some sort of directory structure but still there are no directories on mainframes at all.
A special meaning is associated with the term “high level qualifier”. This means the first qualifier in a dataset name. It can be compared with the entries in the root folder of a UNIX-like file system where each first level folder has some special meaning to the operating system.
Full qualified data set name
Using a dataset name such as TEST.RAFTP.INPUT.X1 is treated by the mainframe system as not fully qualified and will get prefixed internally by a certain high level qualifier as it is defined by the system operator. Usually this is the user ID of the login user. So the above example, assuming the user ID is USER1, will be prefixed by the system and become ‘USER1.TEST.RAFTP.INPUT.X1’. This is called a full qualified data set name.
When using RA-FTP to connect to a mainframe ftp server we have to know that all dataset names are treated like the above example but there is a way to get around the problem by enclosing the dataset name in single quotation marks.
So for our above example TEST.RAFTP.INPUT.X1 equals ‘USER1.TEST.RAFTP.INPUT.X1’ assuming the login users ID is USER1. In many cases a user will need to store or retrieve data having a different high level qualifier then its user ID. If it has sufficient rights to access this data it can be done by using the fully qualified data set name in RA-FTP Job entry fields, so enclosed in single quotation marks.
Note that the access right system (RACF) on the mainframe system regulates access rights to files based on this qualifier system. So the system operator might give access to a certain user or group to ‘SYSTEM.INCLUDE.*’. So the user used to connect to the ftp server must have access rights to the files they want to read or write.
This type of dataset can contain members. The rules for dataset names also apply to this type. A partitioned dataset can be compared to a certain degree to a directory. The members could be seen like files in a directory. But there are limitations for example a partitioned dataset can’t contain other partitioned datasets. The member names can’t be longer than 8 characters and it must start with an alphabetic character (no numbers at first position).
Members of portioned datasets can be access directly by specifying their name enclosed in brackets right after the dataset name. e.g.: TEST.RAFTP.INPUT.PS(MEMB1
). Listing of members can be done by using * as wildcard in the member name. e.g.: TEST.RAFTP.INPUT.PS(*
) will list all members or TEST.RAFTP.INPUT.PS(ME*
) will list all members having a name starting with ME.
This rounds off a very short introduction to the mainframes file system. More details can be found on the internet and in the IBM manuals e.g. ISPF manual. But these basics should be enough to get user’s jobs out of the trouble whenever the target ftp server is a mainframe ftp server. The next chapter will point out the specific differences in common ftp commands.
Specific behavior of a mainframe FTP Server
This unit will provide information on the most important and annoying specifics of the mainframe ftp server based on the very specific MVS file system. It is not a complete list but will be enough to help to get out of problems with standard daily ftp business such as creating directories, up & downloading files.
The Working Directory Dataset Name Prefix is used on mainframe ftp servers instead of a working directory because there are no directories.
- cd (change directory) changes the Working Directory Datasetname Prefix but also opens a partitioned dataset
- ls (list directory) will list datasets using the Working Directory Datasetname Prefix
- pwd (print working directory) will print the Working Directory Datasetname Prefi
Let’s assume we have those datasets available in our root directory and our user ID is USER1. There are written below in the Full Qualified Dataset Name notation.
After we have logged on to the ftp server it will change our working directory name prefix to ‘USER1.’.
If we issue the command pwd
the ftp-server will reply with "USER1." is working directory.
Issuing the command ls
will result in a list of:
Although there are no directories we can use the Change Directory command to extend our working directory name prefix. Issuing the command cd test.city.vie
will do that. Issuing pwd
results in a message: ‘USER1.TEST.CITY.VIE.’ is working directory and ls
will be responded to by the list:
We can use this to limit our list of datasets but we have to keep in mind that there are no directories even when the command Change Directory implies this. In fact the previous list is just a representation of the 2 datasets:
Still there is some use or specific to this as it will allow us to filter. But, it also has the effect of changing the default names for a Get operation. So issuing a command get MAP1
will result in a file named MAP1 on the local system with the content of ‘USER1.TEST.CITY.VIE.MAP1’.
You can use cd ..
to navigate the working directory name prefix backwards. It will change our current working directory prefix to ‘USER1.TEST.CITY.’ and issuing the ls
command will lead to the following response:
Consequently get VIE.MAP1 will result in a file named VIE.MAP1 on the local system having the same content as MAP1.
There is of course one exception which should be mentioned. When using cd on a portioned dataset (those with the members inside) it will kind of “open” that dataset and a subsequent ls will display the members of this dataset. Get operations will then copy the members to the local system and Put operations will add a new member to the dataset.
New partitioned datasets can be created by using the command mkdir
. So mkdir MAPS
will create a partitioned dataset and as our current working directory is still ‘USER1.TEST.CITY’ the fully qualified dataset name is ‘USER.TEST.CITY.MAPS’. An ls
command will respond
We can change into maps by issuing the command cd maps
and copy a local file to it by using the Put command. It will create a member within the partitioned dataset.
This concludes the specifics of the mainframe ftp as those mentioned lead mostly to problems and misunderstandings. There is of course a lot more such as various Quote Site commands to steer the type and sizes of the created datasets, the code page conversion and much more. You can find a lot of information on this topic by searching the internet. IBM websites often have manuals online with complete sets of information about their products and the mainframe ftp server is an IBM product.