Posix scanner version 2.0.1

»  Home

» Software
» Papers & Training
» Java

This README file supercedes the "MPE/iX POSIX Compliance Testing Using Lint Scanner" manual, dated Nov. 1991, available in PDF (B2476-90004.pdf) format
  1. Overview
  2. Contents
  3. Installation
  4. Collection on C sources
  5. Reduction on C sources
  6. Collection on HP-UX executables
  7. Reduction on HP-UX executables
  8. Reviewing the report
  9. (End of Page)

What's New in version 2.0.1 - March 26, 1999

  • lib files reorganized into subdirectories to make it more obvious what's on MPE/iX, HP-UX, etc.
  • lib/HP_UX/hpux_libraries & lib/HP_UX/hpux_system_calls updated for HP-UX 10.20

What's New in version 2.0.0 - March 15, 1999

  • added new ability to scan HP-UX executables

Overview

The POSIX Scanner toolkit is useful to analyze an applicaton you may want to port to the HP3000. In two steps, external functions called by the code are collected and then reduced into a report showing which functions are or are not available on MPE/iX. A sample of the final report is in file report_sample. It is now available in 2 versions: the first scans C source code and the second analyzes HP-UX executables. Note #1: the tool for C source code uses lint on HP-UX to scan C source code. Because lint does NOT work on C++ source code, this tool does NOT handle C++ source code. Note #2: the tool for HP-UX executables uses nm on HP-UX to get symbol information from the executable. If the executable has been stripped of symbol information, this tool cannot report function usage.

Contents

The POSIX Scanner files include:

Installation

Upload sample

ftp> bin
200 Type set to I.
ftp> quote site posix on
200 POSIX command ok.
ftp> cd /tmp
250 CWD  file action successful.
ftp> put mpeincl.tar.Z
200 PORT command ok.
150 File: mpeincl.tar.Z opened; data connection will be opened
226 Transfer complete.
ftp: 993765 bytes sent in 1.03Seconds 963.88Kbytes/sec.
ftp> put mpecont.tar.Z
200 PORT command ok.
150 File: mpecont.tar.Z opened; data connection will be opened
226 Transfer complete.
ftp: 203183 bytes sent in 0.36Seconds 565.97Kbytes/sec.
ftp> put mpebsd.tar.Z
200 PORT command ok.
150 File: mpebsd.tar.Z opened; data connection will be opened
226 Transfer complete.
ftp: 79129 bytes sent in 0.17Seconds 460.05Kbytes/sec.
ftp> put ps.tar.Z
200 PORT command ok.
150 File: ps.tar.Z opened; data connection will be opened
226 Transfer complete.
ftp: 100967 bytes sent in 0.33Seconds 307.83Kbytes/sec.
ftp>

/tmp#ls -l *.Z
-rw-------  1 MGR.KEVENM     KEVENM     79360 May 19 22:08 mpebsd.tar.Z
-rw-------  1 MGR.KEVENM     KEVENM    203264 May 19 22:08 mpecont.tar.Z
-rw-------  1 MGR.KEVENM     KEVENM    993792 May 19 22:08 mpeincl.tar.Z
-rw-------  1 MGR.KEVENM     KEVENM    101120 May 19 22:08 ps.tar.Z
/tmp#uncompress *.Z
/tmp#ls -l *.tar
-rw-------  1 MGR.KEVENM     KEVENM    286323 May 19 22:08 mpebsd.tar
-rw-------  1 MGR.KEVENM     KEVENM    692776 May 19 22:08 mpecont.tar
-rw-------  1 MGR.KEVENM     KEVENM   3813901 May 19 22:08 mpeincl.tar
-rw-------  1 MGR.KEVENM     KEVENM    333388 May 19 22:08 ps.tar
/tmp#  
The POSIX Scanner toolkit is implemented as shell scripts and is usually installed on a UNIX system, typically HP-UX, but doesn't have to be; only a system where lint (to scan C sources) or nm (to scan executables) is found. This is also where the source code or executables are; if not, they must be moved here. (There is a lint, called lclint, on the MPE Freeware tape, but I haven't incorporated it into this tool. Also, lclint does not handle C++ source code.) To install the POSIX Scanner (PS) toolkit, first cd to where you want the ps directory to reside:
$ cd <where ps directory will reside> 
and extract the PS toolkit. The ps.tar archive contains file paths which begin with the ps directory.
$ tar xvf /tmp/ps.tar 
Add this directory to your PATH so you can execute the scripts; assuming you created the ps directory in your home directory:
$ export PATH=$PATH:$HOME/ps 
This allows you to execute the PS scripts from where the source code is. Verify by entering:
$ pscollcsrc 
You should get a usage statement. If you are going to analyze C source code, install the MPE header files; the set for MPE release 6.0 is in tar archive mpeincl.tar:
$ mkdir mpeinclude
$ cd mpeinclude
$ tar xvf /tmp/mpeincl.tar
Also, it's assumed the C sources (and application header files) are already on the system. If you are going to analyze HP-UX executables, you do not need to install any of the header file tar archives. Also, it's assumed the executables are already on the system. Finally, move to where the sources or executables are:
$ cd <C source or executable root> 
If you are going to scan C sources, continue below. If you can going to scan executables, skip down to Collection on HP-UX executables.

Collection on C sources

The POSIX Scanner collection phase for C sources is executed with the pscollcsrc script; it calls lint with the desired options. Lint works like a compiler and has many of the same options as the C compiler. The actual pscollcsrc execution is shown below after first explaining some necessary options.

First, determine if any platform flags must be specified as -D options. This assumes the sources are written for multiple platforms using compile flags. You want to specify the MPE flag(s) if the sources have already been modified for MPE or choose the platform closest or similar to MPE (usually HP-UX). As an aside, if you start modifying the source for changes pertaining to MPE, define a switch named MPE_IX and use #ifdef MPE_IX statements in your sources to identify MPE specific code. It is recommended you NOT use flags __mpexl (defined by the compiler) or _MPEXL_SOURCE (used in header files) since flags starting with '_' (underscore) are reserved for system use. Just use -DMPE_IX as a platform flag on your PS and compile command lines.

Second, determine if any application header files exist. The directories where these are located need to be specified as -I options. Script includeDirs.sh may be useful to identify all the directories in one or more source trees which contain application header files.

Third, note where the MPE header files were installed, to be specified as an additional -I option. Because you are using the MPE header files, it is desirable to NOT use the native system header files. If you are so lucky as to have superuser access, you can make sure the native system header files are not used by moving them:
$ cd /usr $ mv include include_save
Remember to move them back after the collection phase is over.

Fourth, determine any options which are used during compilation besides defines and includes. Most of these can also be specified to lint (check lint's man page for details), but note lint's -c option is NOT the same as the compiler's -c option.

Fifth, specify ALL sources so they are scanned in a single execution. Otherwise, a function in a source file left out will be found undefined. Script sourceDirs.sh may be useful to identify all the directories in one or more source trees which contain source files. Execute the PS collection script:
$ pscollcsrc -v -D<flag>... -I<appl_header_dir>... -I<mpeinclude_dir> *.c >APPL_NAME.psc
specifying the platform flags, the application header directories (if any), the mpeinclude directory, and the sources to scan. The -v option specifies a verbose mode; status messages will be sent to stderr. The output is redirected, in this case, to file APPL_NAME.psc; substitute APPL_NAME with your application name. Expect the linting of your sources to take about the same time as a compilation would; if it takes 30 minutes to compile a set of sources, the linting will take about as long.

Two other scripts starting with pscollcsrc_ demonstrate how the pscollcsrc is called with flags, etc. Or you may find it easier just to call lint directly; just use the lint options used in script pscollcsrc. If lint gives you a "Can't open include file 'x.h'" error, it may be in another header fileset. There's the BSD header files in tar archive mpebsdincl.tar and the contrib/Gnu header files in tar archive mpecontribincl.tar; look at the corresponding .tar.out to determine if the missing header file is there. Review the output to reduce the number of errors to a minimum.

If lint fails on some source with a "Cannot recover from earlier errors, terminating" error, the resulting report will not be as detailed as it could be. So reducing and eliminating errors is VERY IMPORTANT! You may not eliminate all errors or even warnings, but you should try. Warnings about code syntax would imply modifying the code to eliminate them; you can probably skip these kind of warnings. But errors about missing header files, undefined types, or things external to the source files should be resolved. If you are still having trouble reducing the number of errors, you may find an old manual helpful.

The "MPE/iX POSIX Compliance Testing Using Lint Scanner" manual, dated Nov. 1991, is available in Acrobat Reader (B2476-90004.pdf) format. Chapter 5 "Interpreting the Result" doesn't really cover the titled topic, but more focuses on reducing the number of errors from lint. With the addition of the Reduction script described next which really interprets the results, this README file supercedes the old manual. Continue onto the next step - Reduction on C Sources.

Reduction on C sources

The reduction step takes the pscollcsrc or lint output and sorts them according to libraries found on the HP3000. To reduce the pscollcsrc or lint output, enter:
$ psredcsrc -v APPL_NAME.psc >APPL_NAME.rep 
specifying one or more output files from pscollcsrc (in this case APPL_NAME.psc). The -v option specifies a verbose mode; status messages will be sent to stderr. The report is what's written to stdout (in this case APPL_NAME.rep). You will be prompted for a title for the report; specify the application name and version, if known. The next step is to review the report; skip down to Reviewing the report.

Collection on HP-UX executables

The POSIX Scanner collection phase for HP-UX executables is executed with the pscollexe script; it calls nm with the desired options. First, identify any shared libraries the application uses. What the application may call is another application function, not a system function. In order to eliminate these additional application defined functions, the shared libraries containing them must be specified. Enter:
$ pscollexe -v [-L<dir>]... [-l<lib>]... executable... >APPL_NAME.pse
specifying the shared libraries, if any, and the executables to scan. You can use the shared libraries' full name with the -l switch or use the -L & -l switches like the ld link command. The -v option specifies a verbose mode; status messages will be sent to stderr. The output is redirected, in this case, to file APPL_NAME.pse; substitute APPL_NAME with your application name. Continue onto the next step - Reduction on HP-UX Executables.

Reduction on HP-UX executables

The reduction step takes the symbols from the pscollexe or nm output and sorts them according to libraries found on the HP3000. To reduce the pscollexe or nm output, enter:
$ psredexe -v APPL_NAME.pse >APPL_NAME.rep 
specifying one or more output files from pscollexe (in this case APPL_NAME.pse). The -v option specifies a verbose mode; status messages will be sent to stderr. The report is what's written to stdout (in this case APPL_NAME.rep). You will be prompted for a title for the report; specify the application name and version, if known. Continue onto the next step - Reviewing the Report.

Reviewing the report

The report lists the libraries containing the functions your application calls. If you scanned C sources, you should get a report while looks like report_sample; the source file containing the particular function is listed. If you scanned HP-UX executables, the executable containing the particular function is listed. Only libraries containing functions which the application calls are listed. The sample report in file report_sample was done with a source test where functions from every library were called. Your report will usually be much simpler. Listed below are all the possible libraries and categories; the number of functions in each library is also listed:
POSIX functions in libc, but defective: 6
POSIX functions in libc, but return "Function not implemented" error: 8
POSIX functions in libc: 164
POSIX functions in libm: 19
POSIX functions in libpthread: 28
POSIX - some other component available: 8
POSIX functions not available: 127

ISO C functions (not POSIX) in libc: 122
ISO C - some other component available: 7

C Library functions (not POSIX or ISO C) in libc: 59
C Library functions (not POSIX or ISO C) in libm: 19

Other functions (not POSIX, ISO C, or C Library) in libc,
but return "Function not implemented" error (deleted from POSIX): 2
Other functions (not POSIX, ISO C, or C Library) in libc: 5
Other math functions (not POSIX, ISO C, or C Library) in libm: 17
Other PThread functions (not POSIX, ISO C, or C Library) in libpthread: 127

Appletalk functions in libatalk: 65
Berkeley Sockets functions in libsocket: 28
Curses functions in libcurses: 138
DCE functions in libdce: 2846
Motif functions in libMrm: 178
Motif functions in libUil: 230
Motif functions in libXm: 684
NWAPI functions in libnwapi: 398
Other thread functions in libOTHD: 99
Stream functions in libstr: 9
SVID IPC functions in libsvipc: 13
TLI functions in libtli: 71
X11 functions in libX11: 525
X11 functions in libXt: 365
Other components found in MPE: 3

BSD functions in libbsd: 152
HP-UX system calls not available: 275
HP-UX subroutine or library functions not available: 1532
Change for MPE/iX Release 6.0: raise(), an ISO C function (but not POSIX), was defective prior to release 6.0, but is now working correctly. The workaround or wrapper is still included in the POSIX wrappers, in case you need it.

Don't be alarmed by the number of defective or "Function not implemented" functions. All these functions are available and can be downloaded from the Jazz web server jazz.external.hp.com; select Freeware, POSIX wrappers. Many of the not available functions can also be found in the same package.

Finally, if the function is unknown, it is added to the Other functions list at the end of the report. For all functions on the report, the source or executable from which it was called is listed. You should review each entry listed in the Other functions list to determine if it's defined in your code set or somewhere else. If you scanned C source code, lint also flags external variables and structs so you may find some of them in the Other functions list. You just need to know what each entry is; something from a third party product, for example? Or maybe you forgot to include some source files (if you scanned C source) or a shared library (if you scanned executables); if so, start again. Therefore, it is important to review each entry in the Other functions list.

Enjoy.

Send any comments or feedback to: Mark Bixby

Top    JazzInfo    Hosted by 3kRanger.com    email 3kRanger    Updated