HP 3000 Manuals HP 3000 Manuals
The Hierarchical File System
by Craig Fairchild and Rajesh Lalwani
Commercial Systems Division
Overview
This version of MPE/iX provides a hierarchical directory structure. Now
you can create hierarchical directories and place other hierarchical
directories and files under them. MPE/iX 5.0 supports several new file
and record types including the byte-stream record type, pipes, FIFOs, and
symbolic link and device link file types. In addition, filenames can
contain some special characters and can be as long as 255 characters.
This article describes these new 5.0 hierarchical file system (HFS)
features.
Hierarchical Directory Structure
The MPE operating system was originally designed primarily for use in a
department setting. All the users in a particular department are
typically grouped together and placed in one account. Similarly, an
account, such as PAYROLL, could organize its files in groups such as
JAN92, FEB92, and so on. In this model, the directory tree consists of
one or more accounts, with each account containing one or more groups.
Finally, each group has zero or more files in it. Hence, this
file.group.account structure can be looked at as a 3-level hierarchical
structure. The first level has account entries, the second level group
entries, and the third level the actual files.
POSIX, however, has an expanded version of MPE hierarchy. The POSIX
directory structure has a root directory, which is represented by a slash
(/). The root directory may have files and directories under it. The
directories, in turn, have files and directories under them. MPE/iX has
integrated the MPE and POSIX concepts resulting in the new HFS. A typical
MPE/iX directory could look like:
/
|
+--------------------------+--------------------------+
| | |
PAYROLL DEVELOP users
| | |
+------+------+ +------+------+-----+------+ +--------+--+
| | | | | | | |
JAN92 FEB92 MPE dir file PX bruna jeff
| | |
+--+--+ +------+------+-----+ +---+---+
| | | | | | | |
HOURS TAXINFO ledger payable data PRGFILE bin addr_bk
| |
+--+---+ +---+-----+
| | | |
main.c Makefile my_ls down_load
On MPE/iX, the root directory may contain files, directories, and
accounts. For most purposes, accounts and groups are treated as
directories but there are differences. For example, groups and accounts
cannot be purged by the PURGEDIR command. Also, groups and accounts
contain some accounting information associated with them which is not the
case with HFS directories.
Like any MPE system, the accounts contain groups. The groups can, in
turn, contain files or directories. In the above figure, three different
flavors of the integration of MPE and POSIX concepts can be seen.
PAYROLL account has not used any of the new features of POSIX. The
account PAYROLL contains two groups JAN92 and FEB92. The group FEB92
contains just two files HOURS and TAXINFO. Notice that the files in the
group FEB92 even have valid MPE names.
A person familiar with UNIX has created the directory users under the
root directory. This person has placed two directories, bruna and jeff
under users. The directory jeff has a simple file addr_bk and a
directory bin under it. The directory bin has two script files my_ls and
down_load.
The person using the DEVELOP account has used both MPE and POSIX
concepts. The account DEVELOP has two groups MPE and PX, as well as a
file and a directory. This account is apparently used by an application
developer. There are two directories under the group PX, ledger and
payable, probably for keeping source and make files related to the two
modules of the accounting package. The whole application, after
compilation and linking, has been kept in the file PRGFILE. Apparently,
this package keeps the data in the file data.
Here are the highlights of the
MPE/iX directory structure:
* The root directory (represented by a slash /) can have accounts,
directories, or files under it.
* Accounts may have groups, directories, or files under them.
* Groups may have files or directories under them.
* Directories may have files or directories under them.
* Special directories "." and ".." refer to the directory itself,
and the parent of the directory, respectively. The ".." entry of
root (/) is defined to be root (/).
* Only accounts and groups must have valid MPE names. All others
objects can use the portable HFS character set that consists of
A-Z, a-z, 0-9, and three special characters ., _, and -. No name
can start with a hyphen (-), but may start with any other
character. For example, _a_file and 123 are valid HFS syntax
filenames. Also note that these names are case sensitive; AFILE,
Afile, and afile are all unique filenames and can exist at the
same time.
* All names directly below the root, account, and group directories
can be at most 16 characters in length (except account and group
names that must be valid MPE names). All other names can be up to
255 characters long.
* A pathname may be up to 1023 characters long. This implies
a maximum directory depth of 511 levels (for example,
/a/a/a/a/a/a/a/a...). The system does not prevent the creation of
objects whose absolute pathname would exceed 1023 characters, but
may treat these objects in a degraded manner. For example, STORE
will report an error when attempting to access an object whose
absolute pathname is greater than 1023 characters. These objects
can be removed from the system by using the PURGE command and a
relative pathname.
* All file types (except CIR, RIO, and CM KSAM) may be created
anywhere in the HFS directory. Once opened, access to these files
is exactly the same as if they lived in MPE groups. There are no
HFS files, only HFS names.
Current Working Directory
In traditional MPE, the logon group is used to complete unqualified
names. This allows you to type the shorthand name CATALOG rather than
the full CATALOG.PUB.SYS. The logon group was also used as a security
mechanism and for the location to log connect and CPU utilization
information. In Release 5.0, the dual function of the logon group has
been separated into two parts; the current working directory (CWD) and
the logon group. The logon group is still used to define group user (GU)
access, and as the accounting point for connect time and CPU utilization.
However, now all name qualification is done based on the CWD.
The CWD is initialized to be the same value as the logon group when you
logon. Unless the CWD is explicitly changed by you (with the CHDIR
command) it is impossible to tell any difference in behavior in 5.0 and
4.0 systems. The CHDIR command allows you to place the CWD to any
directory that you can access. This means that if you log onto the
PAYROLL account, you can CHDIR /SYS/PUB to set the CWD to the PUB.SYS
group. You can then refer to CATALOG.PUB.SYS as simply CATALOG.
NOTE The CWD does not have any impact on security. It is used as a
naming shorthand only and is not used to determine GU access (which
is still determined by the logon group).
In HFS terminology, a pathname is called absolute if it starts from the
root and specifies all levels until the target object. An example of an
absolute pathname is /SYS/PUB/CATALOG. A pathname can also be relative,
which means that it does not start at the root, but rather from the CWD.
Examples of relative names include CATALOG, ./CATALOG, and sys/notes. In
each of these examples the name was qualified with the CWD location. If
our CWD was /SYS/PUB, then these relative names would be equivalent to
/SYS/PUB/CATALOG, /SYS/PUB/CATALOG (the dot directory refers to the
directory itself, which in this case is the CWD), and /SYS/PUB/sys/notes.
One other property of the CWD that is different from the logon group is
that the CWD is process local. The logon group of a session is identical
for all processes in that session. However, it is possible for each
process in a session to have different CWDs. This means that if a
program is executed and that program changes the CWD to some location, no
other process is affected by that change. If you were to break out of
the program, you would find that the CI's CWD was the same as when the
program was first run.
Multiple Syntax Support
We just saw where the objects such as groups, accounts, files, and
directories can lie within the MPE/iX directory structure. Let us now
see how we can refer to them. MPE/iX has been designed so that the
existing interfaces (such as intrinsics and CI commands) can refer to all
the objects in the directory without changing the way they refer to the
MPE objects such as files in groups. Also, POSIX interfaces such as
POSIX.1 functions and POSIX.2 commands are able to refer to MPE objects
such as accounts and groups and files within them. Conceptually, there
are three name syntaxes: MPE-only, MPE-escape, and HFS.
MPE-Only Syntax.
The familiar MPE objects (e.g., files in groups) can be referred
to by using any of the following forms; FILE, FILE.GROUP, or
FILE.GROUP.ACCOUNT. When the system processes MPE-only names, it first
starts by upshifting the names since all MPE names are uppercase only.
It then qualifies the names:
* When the filename is fully unqualified such as FILE, it is
qualified with the CWD. If the CWD is not in an MPE group, an
error results because the MPE-only syntax requires the name to be
MPE syntax expressible.
* In the second case when the filename is FILE.GROUP, the name is
qualified to the logon account regardless of the location of the
CWD.
* Finally, in the third case when the filename is FILE.GROUP.ACCOUNT,
the MPE name server refers to the file FILE in group GROUP in
account ACCOUNT, regardless of where the CWD is and what the logon
account is, which is the traditional behavior that would be
expected.
HFS Syntax.
All objects on the system, even MPE objects such as accounts, groups, and
files under them can be referred to by the HFS name server. For naming
purposes, the accounts and groups can be treated as directories. Hence,
the file PRGFILE.PX.DEVELOP can be referred to as /DEVELOP/PX/PRGFILE.
When the system processes HFS syntax names, it performs no upshifting
since HFS syntax names are case sensitive. /develop/px/prgfile is NOT
the same file as /DEVELOP/PX/PRGFILE. If the name is relative, then the
system starts its search from the CWD. If the name is absolute, the path
is searched from the root.
Because the HFS syntax is defined by the POSIX.1 standard, extensions
such as the ability to specify lockwords are not possible when specifying
an HFS syntax name. If the file TAXINFO.FEB92.PAYROLL had a lockword on
it, any attempt to access the file specifying an HFS syntax name results
in an error. This also means that no prompting of the name is done for
interactive users when specifying a lockworded file with an HFS name.
The HFS name syntax is not directly available to the CI commands or any
intrinsic (except HPFOPEN). The MPE-escaped syntax allows the CI and
intrinsics to use the HFS syntax (see below). The POSIX.2 shell
(SH.HPBIN.SYS) accepts only HFS syntax filenames, as well as all of the
POSIX.1 functions.
MPE-Escaped Syntax.
The MPE-escaped syntax is the default syntax used by the CI and the file
system. It is a hybrid of MPE-only and HFS syntax that allows
compatibility with the traditional MPE-only syntax as well as the
flexibility of the new HFS syntax. By default all MPE-escaped syntax
names are treated as if they are MPE syntax names unless they start with
a leading "." or "/", in which case they are interpreted as HFS syntax
names. The names are upshifted, and then qualified in the same manner as
MPE-only syntax. The only difference, in this case, is that it is legal
for an unqualified filename to be in a CWD that is not a group. This
allows you to open a file TEXT in the CWD, regardless of where that CWD
is.
The advantage of MPE-escaped syntax over the MPE-only syntax is the
ability to switch, or escape, to HFS syntax. If the first character in
an MPE-escaped name is either a dot (.) or a slash (/), then the system
assumes that the name is an HFS syntax name. This can be done since both
dot and slash are invalid first characters in an MPE syntax name. When
an MPE-escaped name is escaped into the HFS syntax, the system processes
the name according to the HFS syntax rules. No upshifting is done to
preserve the case sensitivity, and the special characters dot (.), dash
(-), and underscore (_) are valid in the name as well.
An example of using the MPE-escaped syntax would be in purging the file
/DEVELOP/PX/data. Since the data file is in lowercase, an MPE syntax
name cannot name the file. If our CWD is /DEVELOP/PX, then either of
these two commands would have the same effect:
:PURGE ./data
or
:PURGE /DEVELOP/PX/data
Note that the use of the leading dot in the example above is only
necessary to escape the name into the HFS syntax. If you ran the POSIX.2
shell, which uses HFS syntax - not MPE-escaped syntax like the CI, you
would not need to provide the leading "./" characters.
Creating/Purging Hierarchical Directories
Hierarchical directories can be created by the intrinsic HPFOPEN, CI
command NEWDIR (refer to the article "New CI Commands" in this
Communicator), or by POSIX.1 and POSIX.2 interfaces.
The disk space used by hierarchical directories that are somewhere below
an MPE group (like ledger and the files main.c and Makefile in the
example directory shown previously), counts toward the file space limits
of the group and account to which they belong. However, for directories
that are not below an MPE group (like users or jeff), system managers
cannot put a limit on the disk space that they can take up. However,
only users with SM capability can create a directory directly below the
root directory. Also, a new CI command DISKUSE can be used to see the
disk space used by various directories.
A hierarchical directory cannot be purged unless it is empty. A
directory is considered empty when it only has the dot (.) and dot-dot
(..) entries in it. A new CI command PURGEDIR has an option ;TREE that
purges the files/directories under the directory recursively and finally
purges the requested directory.
New File Types
This version of MPE/iX provides some new file types. Following is a
brief definition of them. Further information can be obtained from the
POSIX Reference from IEEE or the MPE/iX manual set. Since these
interfaces are based on a commonly used standard, many textbooks are
available that contain a detailed discussion of their use.
Pipes.
A pipe is a nameless file that is typically used for interprocess
communication between parent and child processes. A pipe consists of two
file descriptors that are connected so that data written to one can be
read from the other in a first-in-first-out manner.
FIFOs.
A FIFO is a named pipe. Because a FIFO has a directory name, it can be
used by non-related processes to perform interprocess communication.
Symbolic Links.
A symbolic link is a permanent file in the system directory that points
to another pathname. This is similar to using an MPE/iX file equation
(for example, :FILE FILE1=FILE2) that lives permanently. In this
example, any reference to the name "FILE1" actually opens the file
"FILE2". A symbolic link performs the same function. Please refer to
the article, "Symbolic Links on MPE/iX," later in this chapter for more
detailed information.
Device Links.
A device link associates a directory entry with a logical device number.
It is conceptually similar to issuing a file equation such as :FILE
TAPE;DEV=7. Whenever the device link file is opened, the open is
actually performed on the logical device associated with the device link.
Byte-Stream Files.
With MPE/iX Release 4.5 and subsequent releases, MPE/iX has a new record
type: byte-stream. Byte-stream files are simply a sequence (stream) of
bytes, hence the name byte-stream. The term byte-stream file is
frequently used when talking about files with the byte-stream record
type, even though they are not a new file type. Byte-stream files do not
have any record structure associated with them. One can build a
byte-stream file as follows:
BUILD filename;rec=,,B;disc=100000
Notice that you do not have to specify the record size. This is because
byte-stream files have a record size of 1 byte. B means byte-stream in
the same way as F means Fixed length record file. Also, since a
byte-stream file is implemented with record length of one byte, disc=
parameter would indeed mean the number of bytes in the file.
By convention, applications use the linefeed character (ASCII 10) to
separate records in byte-stream files. Byte-stream files also allow
reading and writing of data in arbitrary chunks. In order for an
application to read logical records, it must read a chunk into a buffer
and scan for the linefeed record separator. This buffering can be
complicated and can also be handled for the application by C Library
routines. Most traditional applications, however, are not written in C
and would not operate correctly with true byte-stream access.
So that the current applications can work correctly with byte-stream
files, intrinsics such as FOPEN and HPFOPEN binds a Variable Record
Emulator type manager with a byte-stream file by default. In such a case
the byte-stream file behaves like a variable record file with the
linefeed character acting as the end of the record marker. If the
application wants to use the file as a byte-stream file, it needs to
specify HPFOPEN option 77 as 2 while calling HPFOPEN.
Miscellaneous
* With this release of MPE/iX, users with SM capability can create
and rename files across account boundaries
* You no longer have to be the creator of a file to RENAME it. If
you can read the old file, purge the old file, and create the new
file, then you can RENAME the file.
* The file label has several new fields such as file_group,
state_chg_time, and hd_link_count. For some files, the file_name
field is empty because the filenames can now be 255 characters in
length. The creator field can contain a fully qualified name of
the creator since the files can now be created by users from a
different account.