HPlogo HP-UX Reference Volume 5 of 5 > d

dld.sl(5)

» 

Technical documentation

Complete book in PDF

 » Table of Contents

 » Index

NAME

dld.sl — dynamic loader

MULTITHREAD USAGE

The dynamic loader is thread-safe.

DESCRIPTION

The /usr/lib/pa20_64/dld.sl program is the PA64 dynamic loader. The /usr/lib/dld.sl program is the PA32 dynamic loader. In programs that use shared libraries, dld.sl is invoked automatically at startup time by exec on PA64 and by the startup file crt0.o on PA32. Identical copies of crt0.o are kept in both /opt/langtools/lib and /usr/ccs/lib directories. The dynamic loader is, itself, a shared library, although it defines no symbols for use by user programs.

Shared Libraries

Shared libraries are executable files created with the -b option to ld (see ld(1)). They must contain position-independent code (PIC) that can be mapped anywhere in the address space of a process and executed with minimal relocation. PIC can use PC-relative addressing modes and/or linkage tables. It is generated by default by the compilers on PA64 and by specifying the +z/+Z options on PA32. See the +help option to ld(1) or the HP-UX Linker and Libraries User's Guide manual for details on writing PIC in assembly language.

Incomplete Executables

An executable program linked with one or more shared libraries is called an incomplete executable.

When creating an executable (a.out) file from object files and libraries, the linker does not copy text (code) or data from the shared library into the output file. Instead, the dynamic loader maps the library into the address space of the process at run time. The linker binds all program references to shared library routines and data to entries in a linkage table, and relies on the dynamic loader to fill in the linkage table entries once the libraries have been mapped. This linkage table serves as a jump table for function calls.

Thread local storage is now supported. The dynamic loader will tally each shared library's thread local storage size as well as the program's thread local storage size. When all libraries are loaded, on PA32 the dynamic loader will either call a thread routine to set the thread pointer and allocate space for the initial thread or will call mmap() and lwp_setprivate() to allocate the space and setup the thread pointer. On PA64, the dynamic loader will invoke an initializer in the system library libc which will do the thread initialization, allocation of the initial thread, and set the thread pointer.

On previous PA32 releases, shared library data items referenced by the program were copied into the program executable file so that the data references could be resolved statically. Beginning with the Series 700/800 10.0 release, references to shared library data from the a.out are included in a linkage table and are resolved at run time.

Loading

An incomplete executable contains a list of path names of the shared libraries searched at link time. At run time, the dynamic loader attaches to the process all shared libraries that were linked with the program. The dynamic loader will attempt to load each library from the same directory in which it was found at link time. It is possible to change the shared library run time search path by specifying a dynamic path list. See PA32 Dynamic Path List and/or PA64 Dynamic Path List.

The text segment of a library is shared among all processes that use it. The data and bss segments are shared on a page-by-page basis. When a process first accesses (reads or writes) a data or bss page, a copy of that page is made for the process.

PA32 Dynamic Path List

There are two was to specify a dynamic path list :

  • by storing a directory path list in the executable using the +b path_list option to ld,

  • by linking the executable with ld option +s, enabling the executable to use the path list defined by the SHLIB_PATH environment variable at run time.

The path list is a list of one or more path names separated by colons (:). The dynamic path list will work only for libraries specified with the -l or -l: options to ld. However, it can be enabled for libraries specified with a full path name using the -l option to chatr (see chatr(1)). If both +s and +b are used, their relative order on the command line indicates which path list will be searched first in compatibility mode. See the +help option to ld(1) or the HP-UX Linker and Libraries User's Guide manual for more details.

PA64 Dynamic Path List

For standard mode libraries (libraries built or linked with ld +std ), the dynamic loader will use dynamic path searching to find shared libraries whose names appear in a library list of the program or loaded shared libraries with no embedded / character. Dynamic path searching is enabled by default for these standard mode libraries or executables. If ld +noenvvar is specified, the dynamic loader will not look at any dyanmic path environment variables to find dependent shared libraries. This limits the dynamic path searching to the value of rpath and the default directories /usr/lib/pa20_64 and /usr/ccs/lib/pa20_64.

For compatibility mode libraries (libraries built or linked with ld +compat), the dynamic loader will only do dynamic path searching for these libraries if they were linked with -l or -l: and one of these were specified:

  • ld +s

  • ld +b

  • chatr +s enable

There are several ways to specify a dynamic path list :

  • By storing a directory path list in the executable using the +b path_list option to ld.

  • By not specifying ld +b and letting the linker set the rpath value to a concatenation of the ld -L path_list followed by the value of the environment variable LPATH followed by the default directories /usr/lib/pa20_64 and /usr/ccs/lib/pa20_64. This is for standard mode shared libraries only.

  • By storing a directory path_list in the environment variables LD_LIBRARY_PATH and/or SHLIB_PATH. For compatibility mode shared libraries and executables, the directory path_list should only be put in the SHLIB_PATH environment variable.

The path list is a list of one or more path names separated by colons (:). The dynamic path list will work only for libraries specified with the -l or -l: options to ld. However, it can be enabled for libraries specified with a full path name using the -l option to chatr (see chatr(1)). If both +s and +b are used, their relative order on the command line indicates which path list will be searched first in compatibility mode. See the +help option to ld(1) or the HP-UX Linker and Libraries User's Guide manual for more details.

The dynamic loader will use these rules when determining which dynamic path list to use:

  • If ld +noenvvar was specified and ld +b and ld +compat were not specified, then the only dynamic path searching that can be done is to look at the path_list in rpath followed by the default directories /usr/lib/pa20_64 and /usr/ccs/lib/pa20_64.

  • If ld +compat was specified and ld +b and ld +s were not specified, no shared library is subject to dynamic path searching.

  • If the ld +compat and ld +b options are not specified, then the path_list in the LD_LIBRARY_PATH environment variable is searched, followed by the path_list in the SHLIB_PATH environment variable, followed by the path_list in rpath, followed by the default directories /usr/lib/pa20_64 and /usr/ccs/lib/pa20_64.

  • If the ld +compat, ld +b, and ld +s are specified, then use the relative ordering of ld +b and ld +s to determine if the dynamic loader should use the path_list in rpath before SHLIB_PATH followed by library as specified in the library list. If ld +b is specified first, use the path_list in rpath first.

The rules change slightly when looking for dependent shared libraries.

  • For standard mode libraries, the path_list in the LD_LIBRARY_PATH environment variable is searched first, followed by the path_list in the SHLIB_PATH environment variable, followed by the value in the parent shared library's rpath, followed by the default directories /usr/lib/pa20_64 and /usr/ccs/lib/pa20_64. The ancestors of a parent shared library may contain a path_list in rpath, but this is ignored when searching for dependent shared libraries of this parent. Only the parent's rpath is used.

  • For compatibility mode libraries, the search is the same as for parent shared libraries, except rpath can be passed from parent shared libraries to child dependent shared libraries to that child's dependents, et cetera.

Binding

The dynamic loader also resolves symbolic references between the executable and libraries. By default, function calls are trapped via the linkage table and bound on first reference. References to data symbols and other absolute address references cannot be trapped. They are bound on the first resolution of a function call that could potentially reference the object.

If the -B immediate option to ld is used, the loader binds all necessary references at startup time. This increases the startup cost of a program, but ensures that no more binding operations will be required later. Thus, better real-time response may result, and the risk of a later abort due to unresolved externals is eliminated.

The fastbind tool can be used to improve the start-up time of programs that use shared libraries (incomplete executables). The fastbind tool performs analysis on the shared library routines and data used to bind the symbols and stores this information in the executable file. The dynamic loader will notice that this information is available, and it will use this fastbind information to bind the symbols instead of the standard search method. For more details refer to fastbind(1) and the +help option to ld(1) or the HP-UX Linker and Libraries User's Guide manual.

Breadth-first Searching

This is only available on PA64. By default, the dynamic loader will do breadth-first searching when binding symbols. If the incomplete executable was linked with +compat or if a shl_load() is being executed, then depth-first searching is used. Breadth-first searching specifies that the dynamic loader will look for symbols starting with the incomplete executable followed by all loaded shared libraries in a left to right order until the symbol is found. For example, the incomplete executable is searched followed by all libraries in its library load list. Then the dependent shared libraries of the first library in the library load list list is searched, followed by the dependent shared libraries of the second library in the list, et cetera.

Depth-first Searching

This is the only search behavior on PA32 and is used on PA64 if doing a shl_load() or if the incomplete executable was linked with +compat. The dynamic loader will search the incomplete executable followed by the first library in its library load list. The first dependent library of this library is then searched, followed by the first dependent of this dependent, and so on. When there are no more dependents, the siblings and their dependents are searched until eventually the second library in the program's library load list is searched, followed by the first dependent of this library, et cetera.

Version Control

Since code from a shared library is mapped at run time from a separate shared library file, modifications to a shared library may alter the behavior of existing executables. In some cases, this may cause programs to operate incorrectly. Two means of version control is provided to solve this problem.

Intra-Library Versioning

This is available on PA32 only. Whenever an incompatible change is made to a library interface, both versions of the affected module or modules are included in the library. A mark indicating the date (month/year) the change was made is recorded in the new module via the pragma HP_SHLIB_VERSION in C, or the compiler directive SHLIB_VERSION in Fortran and Pascal. This date applies to all symbols defined within the module. A high water mark giving the date of the latest incompatible change is recorded in the shared library, and the high water mark for each library linked with the program is recorded in the incomplete executable file. At run time, the dynamic loader checks the high water mark of each library and loads the library only if it is at least as new as the high water mark recorded at link time. When binding symbolic references, the loader chooses the latest version of a symbol that is not later than the high water mark recorded at link time. These two checks help ensure that the version of each library interface used at run time is the same as was expected at link time. Intra-library versioning may be removed in a future release.

Library-level Versioning

The second way for users to version their libraries is by using a new naming convention, libname.n where n is a numeral that is incremented with every new release of the library. When using the new naming scheme, users must specify an internal name for the shared library by using the +h internal_name option to ld when building the shared library. This internal name is recorded in each incomplete executable or shared library that links with the shared library.

At run time, the loader will look at the library list recorded in the incomplete executable file or shared library. For each library in the list that was not an internal name, the dynamic loader will look for a .0 version of the library (e.g. libname.0) to load. If it does not find this version, it will look for the library name that is recorded in the list.

PA32 Explicit Loading and Binding

The duties of the dynamic loader as described above are all performed automatically, although they can be controlled somewhat by appropriate options to ld. The dynamic loader can also be accessed programmatically. The reserved symbol __dld_loc, which is defined in crt0.o, points to a jump table within the dynamic loader. The routines described under shl_load(3X) provide a portable interface that allows the programmer to explicitly attach a shared library to the process at run time, to calculate the addresses of symbols defined within shared libraries, and to detach the library when done.

PA64 Explicit Loading and Binding

The duties of the dynamic loader as described above are all performed automatically, although they can be controlled somewhat by appropriate options to ld. The dynamic loader can also be accessed programmatically. The routines described under shl_load(3X), dlclose(3C), dlerror(3C), dlget(3C), dlmodinfo(3C), dlopen(3C), and dlsym(3C) provide a portable interface that allows the programmer to explicitly attach a shared library to the process at run time, to calculate the addresses of symbols defined within shared libraries, and to detach the library when done.

Global Symbol Table

The global symbol table mechanism is designed as a performance enhancement option. Enabling this mechanism causes the creation of a global symbol table which speeds up symbol lookup, by eliminating the need to scan all loaded libraries in order to find a symbol. This is particularly effective for applications with large numbers of shared libraries. This mechanism is off by default.

The global symbol table is implemented using a hash table. Under this mechanism, whenever a library is loaded (either implicitly or by using dlopen() or shl_load()), the mechanism hashes the library's exports and places them into this table. When a library is unloaded, the mechanism looks up the library's exports in the table and removes them.

The hash table does not contain entries for symbols defined by shl_definesym(). User-defined symbols must therefore be handled separately.

Enabling the mechanism causes dld to use more memory and impacts the performance of the dlopen(), dlclose(), shl_load(), and shl_unload() API calls.

With the global symbol table, the dynamic loader may need to perform a large number of hashing operations to locate symbols. Performing this hash function may cost considerable time, especially when symbol names are very long (C++ programs). To speed up dld, computing hash values can be off-loaded to the linker.

Use the +gst options, +gst, +gstbuckets (32-bit only), +gstsize, +nodynhash (64-bit only), and +plabel_cache, (32-bit only) to control the behavior of the global symbol table hash mechanism. See the ld(1) and chatr(1) commands for information on these options.

With these options, you can tune the size of the hash table and number of buckets per entry to reach a balance of performance and memory use. To maximize for performance, tune the table size for an average chain length of one. For maximum memory use, at the expense of performance, tune the size of the table to minimize the number of empty entries. In general, use prime numbers for the table size. The mechanism provide default values of table size, 1103, and number of buckets, 3.

To get statistical information about hash table performance, set the environment variable _HP_DLDOPTS to contain the -symtab_stat option. This option provideds a message for each library that contains the following information:

  • Operation (load/unload)

  • Name of library

  • Number of exports

  • Number of entries in table with no stored symbols

  • Average length of non-zero chains

  • Calculated performance of the hash table

  • Amount of memory used by the hash table

DIAGNOSTICS

If the dynamic loader is not present, or cannot be invoked by the process for any reason, an error message is printed to standard error and the process terminates with a non-zero exit code.

These errors fall into two basic categories: errors in attaching a shared library, and errors in binding symbols. The former can occur only at process startup time but the latter can occur at any time during process execution unless the -B immediate option is used with ld. Possible errors that can occur while attaching a shared library include library not present, library not executable, library corrupt, high water mark too low, or insufficient room in the address space for the library. Possible errors that can occur while binding symbols include symbol not found (unresolved external), or library corrupt.

When using the explicit load facilities of the dynamic loader, these types of errors are not considered fatal. Consult shl_load(3X), dlclose(3C), dlget(3C), dlgetname(3C), dlmodinfo(3C), dlopen(3C), and dlsym(3C) for more information. On PA64, to see error messages, use the dlerror() routine. This routine will print the last error message recorded by the dynamic loader.

WARNINGS

The startup cost of the dynamic loader is significant, even with deferred binding, and can cause severe performance degradation in processes dominated by startup costs (such as simple ``hello world'' programs). In addition, position-independent code is usually slower than normal code, so performance of a program may be adversely affected by the presence of PIC in shared libraries. However, the advantages of decreased disk space usage and decreased memory requirements for executables should outweigh these concerns in most cases.

There are rare cases where the behavior of a program differs when using shared libraries as opposed to archive libraries. This happens primarily when relying on undocumented and unsupported features of the compilers, assembler, and linker. See the +help option to ld(1) or the HP-UX Linker and Libraries User's Guide manual for more details.

The library developer is entirely responsible for version control and must be thorough in identifying incompatible changes to library interfaces. Otherwise, programs may malfunction unexpectedly with later versions of the library. There is little an application user can do if version control is not handled properly by the library developer. The application developer can usually resolve problems by modifying the source code to use the new interfaces then recompiling and relinking against the new libraries.

By default, most warnings are not reported by the dynamic loader.

On PA32, if you wish to see all of the messages, set the environment variable _HP_DLDOPTS to contain one or more options. The following options are supported:

-warnings

Display additional dynamic loader warning messages. Some of these include:

  • Symbols of the same name but different types, such as CODE and DATA. See the WARNINGS section in ld(1) for more details on this warning.

  • Using certain flags or routines described in shl_load(3X).

-fbverbose

See fastbind(1).

-nofastbind

See fastbind(1).

On PA64, if you wish to see all error messages, set the environment variable DLD_VERBOSE_ERR to true.

AUTHOR

The /usr/lib/dld.sl and /usr/lib/pa20_64/dld.sl shared libraries were developed by HP.

SEE ALSO

System Tools:

aCC(1)

invoke the HP-UX aC++ compiler

as(1)

translate assembly code to machine code

CC(1)

invoke the HP-UX C++ compiler

cc(1)

invoke the HP-UX C compiler

chatr(1)

change program's internal attributes

f77(1)

invoke the HP-UX FORTRAN compiler

f90(1)

invoke the HP-UX Fortran 90 compiler

fastbind(1)

invoke the fastbind tool

ld(1)

invoke the link editor

pc(1)

invoke the HP-UX Pascal compiler

Miscellaneous:

a.out(4)

assembler, compiler, and linker output

dlclose(3C)

unload a shared library previously loaded by dlopen()

dlerror(3C)

print the last error message recorded by dld

dlget(3C)

return information about a loaded module

dlgetname(3C)

return the name of the storage containing a load module

dlmodinfo(3C)

return information about a loaded module

dlopen(3C)

load a shared library

dlsym(3C)

get the address of a symbol in a shared library

shl_load(3X)

load/unload shared libraries

Texts and Tutorials

HP-UX Linker and Libraries Online User Guide

(See the +help option to ld(1))

HP-UX Linker and Libraries User's Guide

(See manuals(5) for ordering information)

© Hewlett-Packard Development Company, L.P.