HP 3000 Manuals

MPE Segmenter Reference Manual

SEGMENTATION STRATEGIES 

Remember that, although the Segmenter is a powerful tool, most
programmers never need to access it explicitly.  In almost all
situations, the segmentation accomplished by the MPE operating system is
quite adequate.  If you decide to control or alter segmentation, it will
usually be for one of the following reasons:

 *  You are involved in program design and are testing different
    procedures or versions of a procedure.  Resegmenting or altering the
    contents of a segment gives you freedom to experiment without having
    to rewrite and recompile your source code.

 *  You have a problem to solve, such as the inadvertent duplication of
    an RL procedure name; or the use of an excessive number of RL
    references, resulting in an RL segment which exceeds your system's
    maximum configured code segment size.

The problems which would make altering segment contents or controlling
segmentation necessary (instead of optional, as it is with program
design) are relatively infrequent.

Because segmentation is so adequately handled by the operating system and
because problems with it are infrequent, your control of segmentation
will rarely have a significant impact on performance.  Resegmentation
will not help if you have major problems with your use of system
resources; and if you want a program with exceptional speed and
efficiency, a slight change in program logic or I/O structure will have
more effect than any amount or kind of resegmenting you can do.

Nevertheless, if you have chosen to control segmentation, you will want
to follow some guidelines to make your management as effective as
possible.  As you use the Segmenter, you will develop effective
strategies suited to your system and the kinds of programs you and other
programmers are writing.  However, there are some general guidelines you
can follow.  An understanding of some of the design characteristics of
the HP 3000 and of how the Segmenter works will help you to make
effective use of these guidelines and to develop your own strategies.

The Operating System Environment 

The Multiprogramming Executive (MPE) environment is a dynamic one where
programs are run on the basis of "processes".  A process is the basic
executable entity of MPE. It is not a program, but the unique execution
of a program by a particular user at a particular time.

When you execute a program, a private data segment called a "stack" is
created for that particular execution.  The stack and the program's code
segments together constitute the process, along with any external
references made by your program code.  For example, if several users
access the BASIC interpreter, a separate process is created for each of
them.  They all use the same code, since there is only one BASIC
interpreter; but each user has a unique data stack created by MPE.

Two MPE components have primary responsibility for managing process
execution:  the process dispatcher and the memory manager.  The
dispatcher allocates CPU time to all the executing processes.  The memory
manager's function is to fit code and data segments into main memory as
they are required.  This operation often requires the dispatcher to
decide which already-present segments it must delete in order to make
space available.

When the time-slice for your process begins, the stack and one code
segment are brought into main memory and control is passed to the
program.  As the program proceeds, it will call procedures which are not
in the current segment.  At this point your process is suspended while
MPE arranges to make the required segment present.  In the meantime, the
dispatcher tries to run the process with the next highest priority which
already has a data stack and any required code segments resident in
memory.  When the missing segment for the first process has been made
present, control is passed back to the procedure which called the
segment, and the second process is suspended.

There is no way for you to tell which code segments will be present in
main memory at any particular moment.  All you can be sure of is that the
memory manager will keep the most popular code segments available and
will overwrite those which are rarely used.  If a process needs an
overwritten segment, another copy of the segment will be brought in from
disc memory.  Frequently-used segments remain in main memory, while those
rarely used are in disc memory most of the time.

To you as a user, calling a procedure seems equally easy whether or not
it is in the current segment, since MPE handles process management
without your assistance.  However, there are system constraints which can
be aggravated by poor segmentation.  One of these is the physical size of
main memory, which limits the number of code segments which can be
present at any one time.  Another is the amount of time involved in
swapping absent segments in from disc memory:  while it may not be
noticeable to you, the time involved does affect the efficient use of
system resources and the execution of your process.  A third constraint
is that many processes are competing for CPU time, and a process which is
well-segmented will execute more smoothly, making better use of CPU time,
than one which makes many calls to procedures in absent segments.

Although processes themselves are created, maintained and deleted by MPE,
you do have control over the following process elements:

 *  Which code goes into which segment.

 *  The size of each segment.

 *  The number of code segments.

 *  The size of the data stack (which you control with the compiler
    rather than the Segmenter).

In almost all circumstances, the first element is the only one you need
to be concerned about:  although your programs are unlikely to approach
the segment size and number limitations, which code you place in each
segment can affect the amount of work the memory manager must do to make
code segments present in main memory for executing processes, thus
affecting how quickly and smoothly the CPU can handle your executing
program.

Segmentation Guidelines 

Your concerns for achieving good segmentation should focus on "locality",
which is the degree to which control remains in the same general area of
code.  This focus doesn't change even in systems with physical memory
space problems.  Indeed, locality becomes even more of a concern in such
systems.

ACHIEVE LOCALITY.  Poor locality means that a program branches wildly and
rapidly from place to place, so that when it is segmented and run, there
will be many external references and control will have to pass frequently
from segment to segment.  Inter-segment transfers which require control
to be passed to a segment not present in main memory are particularly
wasteful of time and disruptive to your process and other processes as
the memory manager reorganizes main memory to make space available for
the requested segment.

To work efficiently, the HP 3000 design needs programs that have good
segment locality.  Good segment locality exists when inter-segment
transfers are minimized:  that is, when a program is segmented so that
once the system is working within a particular segment, it stays there as
long as possible, and when it is out, it stays out as long as possible.
(Note that the degree of locality within a segment is not important.)

When they make segmentation decisions, most programmers intuitively feel
that they will achieve localization if they segment according to
function:  that is, they want to put all the command decoding procedures
together in one segment, all the command executors together in another
segment, and so on.  But you will achieve much better results if you
segment according to temporal considerations:  that is, consider how you
can group your procedures so that when your program is executing, the
system will be able to stay within each segment for as long as possible
and pass control smoothly from segment to segment as the program
progresses.

For example, consider a small utility program which dumps a file to the
line printer in some special format.  Suppose the operator can choose the
name of the file and which of those possible formats to use.  The program
has four procedures:  A, B, C, and D.


Suppose, further, that each dump procedure has a procedure to fetch a
record from its file and a procedure to format a print line:


You would probably be tempted to put all the formatting routines in one
segment and the record-fetching routines in another.  But such functional
segmentation would cause a segment boundary to be crossed twice for every
record dumped.  During this transfer time, your program is doing no
useful work and other processes are disrupted, since they must wait for
the CPU to finish with your process.

If you segment temporally, however, with A in its own segment and B1/B2
together, C1/C2 together, etc., then only two segment boundaries are
crossed for a whole dump.

You can help achieve good locality by being careful, when you are
altering or controlling segmentation, not to place in separate segments
procedures which call each other.  Even earlier, when you are designing
your program and writing source code, being aware of how you divide your
program into procedures will have an effect on how your code is segmented
and thus, ultimately, on locality.  It is possible to write programs that
cannot be correctly segmented.  For instance, any procedure or outer
block must reside wholly within a segment.  If it becomes necessary to
move a block of code into another segment, it will only be possible if
the code is a procedure:  you cannot move an arbitrary set of
instructions and place them in another segment but must move the whole
RBM.

You will also want to be aware of the conventions of your particular
programming language for constructing RBMs from source code and compiling
those RBMs into segments.  Appendices A through F will provide you with
some of the major conventions for each language.  Refer to the
appropriate language reference manual for more help.

Routinely using the ;PMAP option when you prepare your programs will help
keep you aware of each program's environment:  the size of each segment,
which procedures are in which segment, and the names of the externals
called by each segment.  Such information will help you minimize
inter-segment transfers as well as to follow the other guidelines for
code manipulation.  You can find more detailed information about ;PMAP in
Appendix G.

ELIMINATE NON-ESSENTIAL MATERIAL FROM YOUR SEGMENTS.  Minimizing the
amount of unused material in any segment, file, account, etc., is a good
general principle, as well as a way to reduce contention for memory
resources.

You can clean up segments by examining the code within them and removing
code which executes only infrequently.  Very often, this will be code
which does error-handling.  Instead of handling detected errors on-line,
write an error-message generating procedure and call it with a parameter
indicating which message to output.  You can put this procedure in a
separate segment so that it does not clutter up main memory while the
system is doing normal, error-free processing of your program.

As another example, suppose you have a program that does an FWRITE and
then checks the condition code for end-of-file.  If required, the program
executes a somewhat elaborate sequence to extend the file by building a
new one, copying the old one into it, and then purging the old file.  If
this condition is likely to occur only once in every 500 program runs,
there is little point in holding the procedure in main memory with other
procedures which do execute frequently.  You can move this code to an
auxiliary segment and let MPE bring it in only when it is needed.
(Remember that you can only move such code out of a segment if it is a
procedure in itself, rather than a series of instructions which are part
of a procedure.)

If your program will be run from multiple terminals, then the code
segments will automatically be shared by the multiple processes, but each
process will have its own data stack.  If your program design requires
data which is never altered, such as error messages, look-up tables,
etc., put the data in your code so that only one copy will be required
for all processes.  Otherwise, all users who run the program will receive
space in their data stack for the code, thus making inefficient use of
main memory resources.

In the following SPL program, the array MESSG is present in the stack of
every process running the program.

     BEGIN
        BYTE ARRAY MESSGINIT (0:22):="TOO MANY TIMES ENTERED";
        .
        .
        .
          PROCEDURE MESSOUT;
          BEGIN
          BYTE ARRAY MESSG (0:22)
          MOVE MESSG:=MESSGINIT,(23);
          PRINT (MESSG,-23,0);
          .
          .
          .
          END;
           .
           .
           .
        END.

We will use main memory space more efficiently if MESSG only exists while
MESSOUT executes.  In the next example, SPL stores the string in quotes
in the code segment.

     BEGIN
        .
        .
        .
          PROCEDURE MESSOUT;
          BEGIN
          BYTE ARRAY MESSG(0:22);
          MOVE MESSG:="TOO MANY VALUES ENTERED";
          PRINT (MESG,-23,0);
          END;
        .
        .
        .
        END.

This guideline and the accompanying examples illustrate the point that
the steps you take to achieve good segmentation are not limited to moving
material from segment to segment or altering material within segments.
Rather, you need to keep in mind the Segmenter's function and the effect
of segmentation on execution, even during phases of the program
development process that precede the Segmenter's involvement.

Obtaining Machine-Readable PMAPs:  The FPMAP 

The Segmenter's PMAPs, produced by the PMAP option of the -PREPARE and
-ADDSL commands and explained in Appendix G, are printed listings which
show where a program's procedures are located in the code segments of a
program or SL file.  When you prepare a program file or SL segment, you
may also request that a machine-readable form of the PMAP be stored in
the program or SL file.  This machine-readable form of the PMAP is known
as the FPMAP.

FPMAPs are useful to subsystems which wish to refer to code locations by
procedure names and offsets within procedures, rather than by segment
numbers and code segment addresses.  For instance, they allow HP TOOLSET
to set and interpret breakpoints based on source-code locations, and
allow APS/ (SAMPLER) to break down program execution profiles by
procedures rather than by arbitrary code segment addresses.

There are two ways to control whether or not the Segmenter will generate
FPMAPs with your program files or SL file segments.  The first is
explicit, and is done by including either the FPMAP or NOFPMAP keyword on
a -PREPARE or -ADDSL command.  The second is implicit, and involves the
setting of the system-wide and job/session FPMAP flags with the -SETFPMAP
command.  These flags allow you to control FPMAP generation without using
the FPMAP/NOFPMAP keywords on -PREPARE and -ADDSL commands, thereby
providing a means for obtaining FPMAPs without modifying your existing
Segmenter job or command files.

SYSTEM-WIDE FPMAP FLAG.  The system-wide FPMAP flag can be used to force
FPMAPs in all program files and SL segments prepared on the system, and
is also used to set the value of the job/session FPMAP option flag when a
user logs onto the system.  To set the system-wide FPMAP flag, you must
have System Manager (SM) capability.  The system-wide FPMAP flag may
assume one of the following values:

UNCONDITION      Causes FPMAPs to be generated for all program files and
(ON)             SL segments prepared on the system, regardless of the
                 setting of the job/session FPMAP flag or presence of the
                 FPMAP/NOFPMAP keywords on -PREPARE and -ADDSL commands.
                 This value also causes the job/session FPMAP flag to be
                 initialized to ON at the beginning of each job and
                 session.

CONDITION (ON)   Causes the job/session FPMAP flag to be initialized to
                 ON at the beginning a job or session.

OFF              Causes the job/session FPMAP flag to be initialized to
                 OFF at the beginning of a job or session.

When a system is started or loaded, the system-wide FPMAP flag is
initialized to OFF.

JOB/SESSION FPMAP FLAG.  Each job or session also has its own FPMAP flag,
which controls FPMAP generation for -PREPARE and -ADDSL commands
performed within that job or session as follows:

ON               Causes FPMAPs to be generated for all program and SL
                 files prepared in the current job or session, unless
                 overridden by the NOFPMAP keyword on -PREPARE and -ADDSL
                 commands.

OFF              Inhibits FPMAP generation for all program and SL files
                 prepared in the current job or session, unless
                 overridden by either the system-wide FPMAP flag (if set
                 to UNCONDITION (ON)), or the FPMAP keyword on -PREPARE
                 and -ADDSL commands.

For programs compiled by HP TOOLSET, the FPMAP will always be generated
if symbolic debugging is requested, regardless of the FPMAP flags and
FPMAP/NOFPMAP keywords.  When preparing such programs, only the NOSYM
keyword can suppress the FPMAP information.

FPMAPs will increase program and SL file size by 3 to 10 percent, and
will cause the CPU time for preparing a program file or SL segment to
increase by 5 to 20 percent.  Since the FPMAP information is not stored
as part of the code segments themselves, it will have no effect on
program execution performance.  Because of the potential benefit of
having the FPMAP information available for use by other subsystems, it is
a good idea to include it with all program and SL files.

Three intrinsics are provided for programmatic access to the FPMAP:
FINDPMAPNAME translates segment names and procedure names to segment
numbers and offsets; FINDPMAPADDR translates segment numbers and offsets
to segment names and procedure names, and DUMPPMAP extracts the entire
FPMAP from a program or SL file into an external file for use by other
applications.  Each of these uses the external FPMAP record format shown
in Figure 3-1.


          Figure 3-1.  External FPMAP Record Format 

Using the Application Program SAMPLER/3000 (APS/3000) 

APS/3000 is an interactive performance tool for tuning applications on
the HP 3000.  You may wish to use it to facilitate your identification
and correction of segmentation problems.

APS/3000 monitors the execution of software (a single program or the
multiple execution of one or more shared program files) and produces
histograms showing the amount of CPU time spent by various programs, or
portions of one monitored program.  You can study the histograms and then
fine-tune your software by optimizing the execution time of:

 *  All processes associated with a single program file.

 *  A single process.

 *  A segment.

 *  A procedure.

 *  An address range.

In a typical application of this performance tool, the user runs and
monitors software for a period of time, studies APS/3000's histograms,
and learns which code consumes the most CPU time.  After optimizing the
code, the user repeats this process until the software performance is
acceptable.

For resegmentation purposes, APS/3000 will allow you to estimate the
number of transfers of control between segments during program execution.
Because it is a statistical sampling technique, APS/3000 assesses system
operation at pre-set time intervals, rather than continually.  Thus it
will not provide the exact number of transfers, nor can it tell you
specifically what to do to make your segmentation more efficient.
However, it can offer you valuable help in identifying problem areas.

You should always specify FPMAP when running APS/3000 since it
automatically generates valuable information that can assist you in
diagnosing program failures.

For more information, see the Application Program SAMPLER/3000 (APS/3000) 
Reference Manual and User Guide (32180-90001).

MPE/iX 5.0 Documentation