HP Sort & Millicode Information


»	Home

»	HP Sort & Millicode Information Center

	»	Patch Information
	»	What is the issue?
	»	What to Look For
	»	Looking for Large Files on your system
	»	Looking for Long Pointers in Program Source
	»	Summary
	»	Sort & Millicode FAQs

»	Software
»	Papers & Training
»	Java

Introduction

We would like to advise you of a set of situations that Hewlett-Packard has identified with MPE/iX software which, in very rare circumstances, could lead to data corruption on HP e3000 systems running MPE/iX Releases 6.5, 7.0, and 7.5. Hewlett-Packard has software patches for these releases ready to correct these situations, MPENX11 and MILNX10, available at the HP ITRC (http://itrc.hp.com/) or via your HP Support Representative. Though Hewlett-Packard believes very few, if any, customers are at risk, HP strongly recommends that all customers take the following corrective actions:

• Install MPENX11 and MILNX10 at your earliest possible convenience (see the patch descriptions below)
• Your non-HP applications may be impacted: examine the applications running on your systems to determine if they meet the criteria to require a recompile (see the notes on patch MILNX10 below).

November 2nd, 2007 Update: FAQ now available here.

Patch Information

Patch MPENX11 addresses the following issues with MPE/iX:

1) SORT.PUB.SYS and programmatic calls to HPSORTOUTPUT: Customers on MPE/iX Release 6.5, 7.0 or 7.5 who sort 4GB or more of data are at risk. When either interface is used to sort more than 4GB of data, the returned record length could in rare instances be corrupted. If the record length returned is less than it should be then data could be lost.
2) MPE/iX OS millicode handling of long pointer access to large files: When the OS copies data from a long pointer location only one byte from a range is moved if, and only if that range starts six, five, four, three or two bytes from the end of a four gigabyte space and the length of the move transfers all remaining bytes of the space. No other source address or transfer length combinations are affected nor are transfers to such addresses. See the section What is the issue for more information.

Patch MILNX10 addresses potential issues with non-HP Software:

It is possible for non-HP Software to have an issue similar to the MPE/iX OS millicode issue described above and thus be at risk of corrupting data. Addressing this issue may require that non-HP Software be recompiled after installation of MILNX10.

Recompilation is not required for programs reading data exclusively through MPE/iX File System intrinsics like FREAD, FREADDIR or HP compiler library file access routines such as C/iX read(), Pascal/iX read(), COBOL READ, as long as patch MPENX11 is installed. A recompile may be required if customers and ISVs have written code to read data from files four gigabytes and larger using Large Mapped pointers, AND used move_fast or code statements that result in a call to $$lr_unk_unk_long or $$lr_na_unk_long. Then, as in the OS issue above, if data is moved starting at locations six or fewer bytes from the end of a four gigabyte boundary, millicode could silently move less than the requested amount of data. HP recommends that customers and ISVs rebuild any applications which read data from large files via long pointers to include the new version of the $$lr_unk_unk_long and $$lr_na_unk_long routines via the new MILLI.LIB.SYS delivered in MILNX10.

Additional information about these issues can be found in the following sections.

What is the issue?

The impacted code involves the "millicode" routines $$lr_unk_unk_long, which implement the PASCAL predefined function "move_fast" and $$lr_na_unk_long. These millicode routines are also used by HP's Pascal and C compilers to implement data movement operations for addresses referenced by Long Pointers. The following table describes the five combinations of long pointer source address and move length that exhibit the issue.

Source value (in hex)	Length value	Destination Alignment
$SID.fffffffa	6	any
$SID.fffffffb	5	any
$SID.fffffffc	4	non-word aligned
$SID.fffffffd	3	any
$SID.fffffffe	2	any

When these combinations of parameters are passed, the impacted version of the millicode will move only one byte of the range. No other source address, destination address, or transfer length combinations are affected.

What to look for:

Software Supplied by HP

Patch MPENX11 addresses these issues for all HP e3000 FOS and SUBSYS software.

Software written for your own use

The good news for your own applications is that with MPENX11 programs which simply call FREAD, FREADDIR or use a language's specific equivalent functions like COBOL READ, Pascal/iX read() and C/iX read() are not exposed to this issue. To be at risk a programmer must have taken some very specific actions. These are:

1) You must open Large Files -- If you don't use MPE Files which are capable of containing more than 4GB minus 64KB of data then you won't have a issue. See Looking for Large Files on your system for more information.

Note: While TurboImage Jumbo Datasets may contain more than 4GB of data they are not subject to this issue.

2) You must be using long pointers to access data in a file 4GB minus 64KB or larger. To use a long pointer in a user mode application, it must have been written in C/iX or Pascal/iX AND have called HPFOPEN with a unique itemnum. Details:

a) The only HP supplied compilers which support long pointers are Pascal/iX and C/iX. Programs written in other HP languages such as COBOL and Fortran cannot directly create or access long pointers. See Looking for Long Pointers in Program Source section for additional information in recognizing long pointers in your code.
b) To obtain a long pointer to a large file, user mode code must call HPFOPEN using itemnum 87, the "Large Mapped Access" option. Please refer to the MPE/iX Intrinsics manual at: http://docs.hp.com/en/32650-90905/ch07s29.html for more information about Large Mapped Access.

3) You must then move data in these files using Pascal/iX move_fast() or your own byte copy routines which result in calls to $$lr_unk_unk_long or $$lr_na_unk_long in conjunction with the five address & length combinations shown above in "What is the issue?". Calls to other Pascal/iX "move" functions or to HP intrinsics such as HPFMOVEDATA, HPFMOVEDATALTOR or HPFMOVEDATARTOL are all safe as long as you've installed MPENX11.

Note: An application must be taking ALL three of the above listed actions in order to be susceptible to this issue. Applications that do not take all three actions are not susceptible to the issue.

Regardless of the current state of your applications HP recommends that all customers install MPENX11 and MILNX10.

The following sections provide further details to assist customers and ISVs in assessing their software's susceptibility to these issues.

Looking for Large Files on your system

Large Files are defined as files built with a file limit allowing more than 4GB minus 64KB of data to be stored. In assessing your risk we advise you to take the following steps.

1) You can look for Large Files that are currently on your system in a number of ways. See http://docs.hp.com/en/30216-90291/ch03s01.html for details about changes to HP intrinsics which allow programmers to gather information about Large Files. HP has written a tool called FILECHEK that will look at all PERMANENT files on your system and determine if they are susceptible to the issues.

CAUTION: Simply running this tool is not sufficient to determine if your applications and files are susceptible to this issue. This tool can only detect files currently on your system which exist in the MPE/iX PERMANENT domain. This program does not detect files built in a JOB or SESSION's TEMPORARY domain.

For more information on FILECHEK click here.

2) You should monitor your system for Large Files usage including those built for temporary use by turning on MPE/iX System Logging type 105 (NM File Close) records. Type 305 records will be created whenever any Large File is closed. Type 105 and type 205 records will also be generated but can be ignored as they do not indicate the presence of a Large File. See Chapter 6 of the "System Startup, Configuration, and Shutdown Reference Manual" http://docs.hp.com/en/32650-90907/ch06.html for more information about MPE/iX System Logging.

CAUTION: Enabling the logging of NM File Close records may cause System Log files to be filled and created at a faster than normal rate -- care should be taken that adequate disk space is maintained on LDEV 1 or serious issues can result.

If you do find large files or see type 305 records in your system logs you will need to determine the applications which are using these files and investigate their susceptibility these issues; see "Looking for Long Pointers in Program Source" for information on evaluating your application code. Remember patch MPENX11 addresses these issues for all HP e3000 FOS and SUBSYS software.

Looking for Long Pointers in Program Source

You should scan the source code of your applications for possible long pointer usage to determine your susceptibility to these issues. The only HP supplied compilers which allow "long pointer" usage are C/iX and Pascal/iX. COBOL, Fortran and other HP supplied languages do not allow long pointer usage; programs and library functions written in those languages require no direct changes.

To obtain a long pointer to a large file, user mode code must call HPFOPEN using itemnum 87, the "Large Mapped Access" option. HP recommends that any application written in C/iX or Pascal/iX making a call to HPFOPEN with itemnum 87 be recompiled and relinked after the installation of MILNX10 in order to ensure the new millicode routines are used.

Here are two code fragments which illustrate how a long pointer obtained from HPFOPEN itemnum 87 might be declared in C/iX and Pascal/iX:

C/iX


/* long pointers are declared with "^" rather than "*" */

char ^long_file_data_ptr;

Pascal/iX


. . . 

{ long pointers are declared with $extaddr$ compiler directive } 

long_file_data_ptr   : ^ $extaddr$ array [1..recsize] of char; 

{ globalanyptr type is a long pointer } 

global_file_data_ptr : globalanyptr;

{ anyptr type is a long pointer, could have an issue later } 

any_file_data_ptr    : anyptr;

Remember that simply having a long pointer declared does not automatically mean you are susceptible to these issues. These pointers must be used to move data from a large file via calls to move_fast() or other compiler statement which resolve to $$lr_unk_unk_long or $lr_na_unk_long order to be susceptible.

Summary

In summary, patches MPENX11 and MILNX10 address these issues for all HP e3000 FOS and SUBSYS software. In house applications developed in C/iX or Pascal/iX will require addition analysis to determine if they should be recompiled (as described above in "What to look for").

Questions about other applications should be directed to the software support provider for that application. If you have questions or require help, please contact your local HP support representative or sales office.

Top JazzInfo Hosted by 3kRanger.com email 3kRanger Updated