HP 3000 Manuals HP 3000 Manuals
Link Editor/XL
by Cary Coutant and Brad Ahlf--Information Technology Group
Hewlett-Packard has developed a new Link Editor for MPE XL to take
advantage of HP Precision Architecture. The Link Editor replaces the
Segmenter for all Native Mode program development. For complete
information about the Link Editor, consult the HP Link Editor/XL
Reference Manual (P/N 32650-90030).
CONCEPTS
Before discussing the new Link Editor/XL functionality or migration from
MPE V systems, it will be useful to discuss some of the new concepts and
terminology, which were introduced with the MPE XL system.
Object Modules
The MPE XL system introduces a new object file format and three new kinds
of object files: Relocatable Object files, Relocatable Libraries, and
Executable Libraries (file codes NMOBJ, NMRL, and NMXL). The first,
relocatable object files, are produced by the compilers, while the
libraries are produced and maintained by the Link Editor. Each is based
on an atomic unit called an object module, of which there are relocatable
and executable varieties, as described below. As the names imply,
Relocatable Libraries contain only relocatable object modules, and
executable libraries contain only executable object modules.
Object modules produced by the compilers are called relocatable object
modules, and generally represent the compilation of a corresponding
source module. Currently, all MPE XL compilers treat the entire source
file (containing one or more procedures) as a source module and therefore
create a single, indivisible object module within a relocatable object
file.
The Link Editor can combine several relocatable object modules into a
single larger one, and can link the result, producing an executable
object module. The Link Editor cannot, however, divide an object module
into smaller pieces. Executable object modules have the same format as
relocatable object modules, except that they have been linked, and
therefore have been bound to absolute addresses (or, more correctly,
absolute offsets within a space) after combining the object modules. Any
unresolved external references remaining after linking are summarized in
an external reference table for processing by the MPE XL loader when the
program (NMPRG) is run.
Locality Sets
To maintain execution efficiency in a paged virtual memory system,
programmers can organize code into locality sets. These inherit, from
the MPE V concept of segments, the property that the loader treats an
entire locality set as a single unit, so that procedure calls within the
same locality set will not suffer from a page fault. In addition,
intralocality procedure calls can often be implemented with a short
branch instruction, resulting in additional savings. A short branch has
a range of 256KB in either direction from the point of call, and costs
only a single machine cycle. A long branch, however, can cost three or
four machine cycles. Since the Link Editor chooses which length branch
is needed rather than the compilers, the use of locality sets can
significantly improve the size and speed of programs.
Relocatable Libraries
The Link Editor can search several Relocatable Libraries and include
selected library routines in the same space as the main program (contrast
this with the Segmenter, which could search only one Relocatable Library,
including the selected routines in a single separate code segment). Note
that an entire module containing any referenced routines will be included
when a Relocatable Library is searched. Like the Segmenter, the Link
Editor builds, maintains, and lists the contents of Relocatable
Libraries. Among the new functions of the Link Editor are copying object
modules from one Relocatable Library to another, and extracting object
modules from a Relocatable Library into a new object file.
Executable Libraries
Executable libraries are in function to the Segmented Libraries of MPE V.
They consist of one or more executable object modules that may each
contain one or more procedures. They are bound to the user's program at
program load time, so they do not occupy any space in the program file,
and the code is shared among all processes using the library. Procedure
calls between executable object modules are routed through an External
Reference Table (XRT), which is a direct analog to the MPE V Segment
Transfer Table (STT).
Relocatable object modules are bound to fixed addresses at the time they
are added to an executable library. An implicit link occurs to
relocatable object modules to make them executable whenever an ADDXL
command is executed. To avoid any need for code relocation at program
load time and also to allow the desired code sharing, each executable
library is assigned to a unique space. The external call, therefore,
must also save and change the currently executing space, and a return
from an external call must restore the previous code space. This is also
similar to MPE V, where the PB and PL registers were changed during an
external procedure call.
The differences between the two architectures result in a subtle
difference between segmented libraries and executable libraries: there
is no direct analog to MPE V segments on MPE XL. On MPE V, segments are
responsible for achieving program locality and provide the foundation for
load-time binding. These two properties of a segment have been
separated. Program locality is now achieved through the use of locality
sets and the paged virtual memory system of MPE XL, while load-time
binding is based on executable object modules. The new $LOCALITY
directive associates a given procedure with a particular locality set,
but does not imply any partitioning of the code into architectural units.
(For source code compatibility, the $SEGMENT directive has the same
effect as $LOCALITY; it should be realized that not all the effects of
segmentation will be felt.) The physical partitioning of code occurs
through creation of executable object modules, each corresponding to a
unique code space.
It may be natural to create one executable object module for each
locality set, thus preserving the logical association between segments
and locality sets. It is equally valid to create an executable object
module that contains many locality sets. Unlike segments, locality sets
have no size restrictions, so there is more freedom in assigning
procedures to locality sets. Likewise, since segments are not required
for memory management, there is no compelling reason to keep executable
object modules small. Calls between locality sets may suffer some
penalty due to nonlocality as mentioned earlier, but an external call
(indirect through the XRT) is not necessary.
A program file is just a special case of an executable library: it
always contains a single executable object module and it has a program
entry point. Since it has only one executable object module, external
calls are used only to call routines in executable libraries.
Some new features related to executable libraries are available in MPE
XL. Users can specify at link time which executable libraries should be
searched at program load time and in which order they should be searched.
Executable libraries may be given any legal MPE file name. These two
features simplify the use of executable libraries, permitting users to
create many libraries and build programs that use the libraries without
requiring any additional syntax on the RUN command. Another significant
feature is that static global-data areas can now be defined in executable
libraries and can be shared by procedures within that executable object
module. There is no way, however, to share data between separate object
modules except through procedure and function arguments. The Link Editor
also allows a user to copy an object module from one executable library
to another, which was not possible with the Segmenter.
LINK EDITOR FUNCTIONS
The primary purpose of the Link Editor is to produce executable programs
given relocatable object files. This process is called linking, and the
LINK command performs the task (it is analogous to the Segmenter PREPARE
command). The other important function of the Link Editor is to build
and maintain relocatable and executable libraries. Most of the remaining
Link Editor commands are devoted to this task.
Linking Programs
The linking process involves three distinct, but interdependent, stages.
First, the Link Editor combines all of the input relocatable object
modules into a single, larger object module. Second, while combining
modules, it relocates all the addresses in the input object modules to
reflect the actual addresses assigned by the Link Editor. Third, it
resolves external references between object modules and searches
Relocatable Libraries.
At compile time, if a reference to a symbol (such as a procedure or
global variable name) that is not defined by that compilation unit is
encountered, the compiler marks that symbol as imported (or unsatisfied).
Similarly, all symbols that are defined by that compilation unit and
designated as global in scope are marked as exported (or universal). At
link time, the Link Editor expects to find an exported symbol in exactly
one module for every unique symbol that is imported from any other module
or modules.
The Link Editor can conditionally include object modules from Relocatable
Libraries, depending on whether they export any symbols that have been
imported by any other modules; this feature is called library searching.
The Link Editor can also unconditionally include all object modules from
a Relocatable Library; this occurs if the library name is listed among
the input files to the LINK command instead of among the Relocatable
Library files.
If the Link Editor finds more than one exported symbol with the same
name, an error is reported, since it is likely that two object modules
have exported two different procedures or global variables with the same
name. If an imported symbol remains unresolved, the Link Editor will
take one of two actions. If the symbol is a data symbol, an error is
reported, since there is no load-time data binding in MPE XL. If the
symbol is a code symbol, the Link Editor assumes that the symbol will be
found by the loader in an executable library. To indicate this to the
loader, a small sequence of code, called an import stub, is provided and
an XRT entry is allocated for that symbol. The import stub contains the
code necessary to make an external call through the XRT entry, which the
loader is expected to fill in at program load time after it has searched
the appropriate executable libraries.
To illustrate the use of the LINK command, a subset of its syntax is
shown below.
LINK
[FROM=file [,file]... ]
[;TO=dest_file ]
[;RL=rl_file [,rl_file]... ]
[;XL=xl_file [,xl_file]... ]
[;CAP=cap_list]
[;PARMCHECK=check_level]
[;NODEBUG]
[;MAP]
[;SHOW]
As one can see from the syntax, all parameters are optional. The
simplest command "LINK" links a relocatable object module from $OLDPASS
and produces a program file in $NEWPASS. Among the new features found in
the Link Editor LINK command are: naming several input files (FROM=),
naming several Relocatable Libraries (RL=), specifying a search order for
executable libraries (this is stored in the program file for use by the
loader), overriding the default parameter type checking level, omitting
all symbolic debugging information from the program file, printing a
symbolic map of the program, and displaying the names of all object
modules as they are being processed during the link.
Relocatable Libraries
It has already been mentioned how Relocatable Libraries are used during
linking. The Link Editor also contains a repertoire of commands to build
and maintain Relocatable Libraries. These commands are BUILDRL, ADDRL,
COPYRL, PURGERL, EXTRACTRL, HIDERL, REVEALRL, LISTRL, and CLEANRL. In
addition, the command RL selects a current Relocatable Library that
serves as a default for most other commands; the command SHOWRL displays
the name of the current Relocatable Library.
A Relocatable Library is built with the command BUILDRL. Object modules
are added to it with the commands ADDRL (to add modules from compiled
object files) and COPYRL (to add modules from other Relocatable
Libraries). The ADDRL command is capable of adding many object modules
individually to the Relocatable Library, or combining them into a single
larger object module that is added to the library. The COPYRL command
can select one or more object modules from one library based on exported
entry points, locality sets, or module names.
The PURGERL and EXTRACTRL commands can select one or more object modules
from a library based on the same selection criteria as COPYRL, and purge
them from the library or copy them into an object file. The LISTRL
command prints a list of modules and the symbols imported and exported by
each module. The CLEANRL command improves the efficiency of library
searching by compacting the library symbol tables; this also reduces the
amount of space required by the library file.
The HIDERL and REVEALRL commands are similar to the HIDE and REVEAL
commands in the Segmenter. As the names imply, they operate on
Relocatable Libraries, but their effect is only evident when the object
modules are added to an executable library. In the Segmenter, hidden
symbols had global scope within their segment, but were not visible from
outside that segment. In the Link Editor, hidden symbols are invisible
only from outside their executable object module. Since object modules
from a Relocatable Library are included in the main program's executable
object module at link time, hidden symbols in a library are still visible
to the rest of the program. Therefore, the HIDERL command (or the
equivalent compiler directive, $OPTION INTERNAL) is ONLY useful to hide a
procedure entry point that is to be placed into an executable library.
Executable Libraries
Executable libraries are built and maintained by a nearly equivalent set
of commands as Relocatable Libraries. These commands are BUILDXL, ADDXL,
COPYXL, PURGEXL, LISTXL, and CLEANXL. The command XL selects a current
executable library that serves as a default for most other commands; the
command SHOWXL displays the name of the current executable library. For
the most part, these commands function just like their relocatable
library counterparts.
The ADDXL command is of particular interest in how it differs from the
Segmenter ADDSL command. Like the ADDRL command, it offers the option of
adding many object modules individually (creating one executable object
module for each relocatable object module), or merging them all into one
executable object module. In the latter case, the Link Editor can even
search Relocatable Libraries while building the executable object module.
Essentially, the ADDXL command differs from the LINK command only in that
it does not expect a program entry point, and that it can add more than
one object module to an executable library.
With virtually no limit on the size of a code space, the question is
raised of how large an executable module should be. In general, there is
no significant performance differences between an executable library with
a single large module and an executable library with several medium
modules. Executable libraries with many small modules should be avoided
because an increased number of external procedure calls would be
necessary and this would significantly impact performance. It is
therefore logical that minimum executable module size should be
determined based on code locality. The primary consideration would then
become one of library maintenance. It is probably easier to group
library procedures by function, and include each group in a separate
executable module, so that updating an executable library involves
replacing one of several modules instead of the entire library.
Since Relocatable Libraries can be searched to produce an executable
object module, it should be noted that it is not always appropriate to do
so. If the routines in the Relocatable Library are expected to be useful
in several distinct executable modules, it would be more desirable
either: (1) to add the entire Relocatable Library to the executable
library as a separate executable module; or (2) to link the entire
Relocatable Library (rather than search it) when adding one of the
executable modules that depend on it. In the second alternative, the
Relocatable Library routines would be an internal procedure call for the
one module, and an external procedure call for the others. It is still
possible, however, to search the Relocatable Library for each executable
module, but all the entry points in the Relocatable Library should be
hidden to prevent duplicate symbol errors. With hidden symbols, each
executable module has its own private copy of just those library routines
that are needed within that module.
Other New Features
The Link Editor also offers a thorough help facility through the HELP
command from within the Link Editor subsystem. The HELP facility is
identical to the one used by the Command Interpreter(CI) and other MPE
subsystems. It contains help screens describing the syntax and
parameters and provides examples for every Link Editor command.
Like the new Command Interpreter, the Link Editor also keeps a listredo
stack of the last 20 commands entered. Users can do, redo, and edit any
of the commands in this list.
Several Link Editor commands (e.g., LINK, ADDRL, ADDXL) allow the user to
specify a list of files for input as object files, or as relocatable or
executable libraries. In any of these situations, an ASCII file can be
prepared in advance containing the list of files, and this "indirect"
file can be specified, preceded by a "^" , in place of the list of files.
The Link Editor will read this list of files as if they had all been
typed on the command line.
Two additional convenience features are also available. First, any MPE
XL command may be executed from the Link Editor by preceding the command
with a colon. Second, any Link Editor command may be executed directly
from the CI by running LINKEDIT.PUB.SYS and including the entire command
in the INFO string. In this mode of use, the Link Editor terminates
after executing the command.
MIGRATION
The process of learning to use the Link Editor after becoming accustomed
to the Segmenter is more than just learning the new command syntax.
There are many similarities between the two products, but the significant
differences and the potential pitfalls of drawing parallels are discussed
below. This section ignores Compatibility Mode, where the MPE V
Segmenter is available and functions exactly as it has on all previous HP
3000s.
Replacing USL File Functionality
Probably the first difference that an MPE V programmer will notice is
that MPE XL does not have User Subprogram Library(USL) files. The MPE XL
compilers, instead of placing object modules into a USL file, now create
new object files (NMOBJ) with each compilation. This change reflects the
observed tendency of many MPE V programmers to use separate USL files for
each source file, combining them only in a long Segmenter script with
lots of AUXUSL and COPY commands. Since the LINK command now accepts
many file names (and even accepts "indirect" files that contain a list of
file names), it is now much more convenient to develop programs using
these techniques.
It is still possible to use Relocatable Libraries as a substitute for USL
files, however. With a UDC or command file, users may automatically add
the object file resulting from a compilation to their pseudo-USL. The
Relocatable Library thus produced can be named as an input file in
the LINK command, and the Link Editor will link the entire file
unconditionally (as opposed to searching it if it were named as a library
file). In this sense, the Relocatable Library behaves just like a USL.
Relocatable Libraries(RL), however, do not support version management.
The Segmenter CEASE and USE commands do not have any equivalent in the
Link Editor, and compilers do no version management as new object modules
are created. Currently, compilers always create a new object file,
possibly overwriting an old one, as if a $USLINIT option were used.
Because of the error-prone nature of the version management feature (old
versions of an object module were denoted by a chronological index, with
no form of user annotation available), and the increasing use of (more
effective) source-level version control systems, this Segmenter feature
was chosen for obsolescence.
Another significant difference between Segmenter and Link Editor use
results from the fact that MPE XL compilers do not maintain a USL. As
mentioned earlier, the compilers currently create one relocatable object
module for an entire compilation, whereas MPE V compilers created one RBM
per subprogram in the compilation. Because of these facts, there is no
practical way to do a subset compilation (a variant of the $SUBPROGRAM
directive, where a subset of procedures in the source file is named in
the directive). The only successful way that such a feature could be
used would be to compile disjoint subsets at all times. Since a
relocatable object module, like an RBM, is an atomic structure, there
would be no way of replacing a subset of the procedures from an earlier,
full, compilation with the new versions compiled in a subset compilation.
A full module replacement always necessary.
Also because of the fact that MPE XL compilers place all procedures in a
single object module, the Link Editor can copy and purge procedures only
by copying and purging ENTIRE OBJECT MODULES. For example, even though
the PURGERL command allows the user to specify an entry point name, the
entire object module that contains that entry point is purged, including
any other entry points contained within that module. Therefore, unless
all source files contain only a single procedure each, the Link Editor
PURGERL command is not an exact replacement for the Segmenter PURGERBM
command.
Relocatable Library Differences
Relocatable libraries are not used used on MPE V, mostly because of the
restrictions associated with them: a program could be linked with at
most one RL, and all modules included from the RL were placed in a single
separate segment. Those who used RLs on MPE V should have no trouble
using them on MPE XL. Those who used USLs because RLs were too limited
should probably now use RLs on MPE XL. As far as compatibility, aside
from additional functionality, the only noticeable difference is that no
separate segment is created. Therefore calls to procedures from a
Relocatable Library can be just as fast as internal procedure calls.
The Segmenter ADDRL command can be translated directly to a Link Editor
ADDRL command by ignoring the new features. If USLs are being emulated
with RLs, however, the Link Editor COPYRL command becomes the appropriate
Link Editor command to use. Since the ADDRL command requires one
parameter, an object file name, the COPYRL option to select by entry
point name or module name would be a translation. Module names are
selected from the source file names and entry point names are from
procedure names and are case sensitive.
It was mentioned above that the MPE XL compilers create one relocatable
object module for an entire compilation, and that this creates some
differences in perceived behavior between the Segmenter and the Link
Editor regarding USL files. It also affects the behavior of Relocatable
Libraries at link time: the Link Editor includes an ENTIRE OBJECT MODULE
in the program file even if only one procedure in that module is
referenced by the program, since object modules are indivisible. This
difference could affect more than just the size of the program file; the
user program may define an entry point that conflicts with one of the
extra procedures that just happens to share an object module with a
procedure that was needed. In such a case, a duplicate symbol error
occurs when the link would succeed if each procedure had been in a
separate module. If this problem does indeed occur, it would be
necessary to recompile each library procedure separately (that is, place
each one in a separate source file), and add the resulting object modules
individually to the library. This technique will produce a Relocatable
Library equivalent to the MPE V USL.
Users who did use RLs on MPE V will probably want to modify their source
code over time to take advantage of locality sets that are available with
RLs on MPE XL. In addition, RL management should become easier since
several libraries can be used for different purposes instead of
maintaining a single library for potentially several applications.
Segmented Libraries versus Executable Libraries
There is little perceivable difference between MPE V segmented libraries
and MPE XL executable libraries. An executable object modules possesses
all the significant properties of a segment with regard to the functions
of a segmented library. In particular, an executable object modules and
a segment are both indivisible, so a reference to one procedure results
in the entire module or segment being bound to the program at load time.
For the user who is making a straightforward translation from Segmenter
usage to Link Editor, a series of ADDSL commands that add the contents of
a USL into an SL can be replaced by a single ADDXL command with the MERGE
option to add an entire Relocatable Library or set of object files into
an executable library. The MERGE option causes all the input modules to
be placed into a single executable object module; there is little to be
gained by adding more, smaller, modules to the library. It is
reasonable, however, to translate segments into locality sets, and a more
literal replacement for an ADDSL command would be an EXTRACTRL using the
locality set selection (assuming an RL is being used), followed by an
ADDXL to add the object file just extracted to the library.
PREPARE versus LINK
Most of the parameters for the Segmenter PREPARE command are either
available in the Link Editor LINK command or meaningless on MPE XL. Those
that are meaningless can simply be ignored. The only significant
difference in command operation is the default object file that is to be
linked: the PREPARE command always uses the current USL, while the
default FROM file for the LINK command is always $OLDPASS (which is the
same as the default for the MPE V PREP command in the CI). If an RL is
being used as a USL replacement, the PREPARE command can be replaced by a
LINK command specifying that file as the FROM file.
HIDERL and REVEALRL
As mentioned earlier, the HIDERL and REVEALRL commands themselves
function just like the Segmenter HIDE and REVEAL commands, in the context
of using an RL file. The effect of hiding a symbol, however, can be
different if there is not a one-to-one correspondence between segments
and executable object modules. If exactly the same effect is necessary,
then the technique presented above for translating the ADDSL command
literally should be used.