MPE/iX Large Files Overview
MPE documents | |
 Complete PDF | |
| Table of Contents | |
| Index |
Commercial Systems Division
Introduction
This document describes the changes to the MPE/iX operating system as a result of the initial addition of Large File functionality. In this document, Large Files are defined as any file greater than 4GB - 64KB (4,294,901,760 bytes) which was the maximum file size prior to release 6.5. For the sake of brevity, this value is referred to somewhat inaccurately as "4GB" for the rest of this document.
Overview
The addition of Large File support to MPE/iX increases the maximum file size supported by MPE/iX from the former limit of 4GB to 128GB for release 6.5. In the future, additional increases in supported file size can be expected as customer needs demand. In this first release of Large File support, two types of files are supported for file sizes beyond 4GB. These are ordinary, fixed length record files, and a new type of KSAM file, KSAM64. A phased approach to Large File support has been taken to get Large File functionality to market in the quickest possible time frame. Several customer surveys have indicated that these two file types are the most needed for Large File support. Large File support of KSAM files requires the addition of yet another KSAM file type. Due to the internal structure of CM KSAM and KSAM XL files, these file types cannot support file sizes beyond 4GB. The implication of this was that in order to meet our customer demand of Large File KSAM support, a new KSAM file type needed to be defined. This new KSAM file type is the KSAM64 file, or file type 7. KSAM64 files have an improved, generalized internal structure that allows them to support file sizes from one record to the maximum object size of 128GB. KSAM64 files are externally identical to KSAMXL files (and support all CM KSAM intrinsic calls as well). Large File support has been added to MPE/iX as seamlessly as possible. Large File support has been implemented within the OS so that Large Files are available on all HP 3000 models. Large Files are created by simply specifying a file limit which is sufficiently large. This can be done interactively through the BUILD or FILE commands, or programmatically through the FOPEN and HPFOPEN intrinsics. Opening an existing Large File is as transparent as opening any other file. All system intrinsics will work on Large Files in the exact same manner as any normal sized file. As a matter of fact, if your program accesses files through MPE/iX intrinsics, it is highly likely that there will be no need to make any changes at all to operate on Large Files. A large part of the ease of Large File use lies in the fact that access to files in MPE/iX is done through a record-oriented interface. All interfaces to the OS still deal with signed 32-bit integers to indicate record numbers. This means that there are really two factors that serve to limit maximum file sizes, the maximum number of records in a file, and the maximum object size supported on MPE/iX. The following table shows how the record size of an ordinary fixed-length record file can affect the maximum size of file.| Record Size | Maximum # of Records | Maximum Object Size | Maximum Resulting File Size |
|---|---|---|---|
| 1 byte | 2,147,483,647 | 128GB | 2GB |
| 40 bytes | 2,147,483,647 | 128GB | 80GB |
| 60 bytes | 2,147,483,647 | 128GB | 120GB |
| 80bytes | 1,717,986,918 | 128GB | 128GB |
About this Document
This document is intended to describe all the external functions, features, and changes that have been introduced with Large File support. The document itself is broken into four sections:Large File Compatibility Issues. This section covers the small number of issues that could arise when accessing Large Files.
Changes to the Command Interpreter. This section describes the enhancements that have been added to several CI commands to support Large Files and KSAM64 files.
New and Enhanced Intrinsics. This section reveals the upgrades to several existing intrinsics for Large File support, as well as introducing a new set of intrinsics to aid in user-mapped file access.
Miscellaneous. This section discusses the impacts of Large Files on various subsystems and utilities, and covers other miscellaneous Large File topics.
Summary of Large File Compatibility Issues
This section covers the compatibility issues that users need to be aware of when dealing with Large Files. In most cases applications and users are unaware that the file that they are operating on is a Large File. Even though Large Files are highly transparent, there are some rare situations that can produce results that are either incompatible or in error when dealing with Large Files instead of normally-sized files. The following sections highlight and describe the issues that can affect access to Large Files.
User-mapped Access to Files
MPE/iX currently supports two types of user-mapped access to files; short-mapped access and long-mapped access. Short-mapped access provides a 32-bit pointer for a user application to directly access a file. There are several limitations to short-mapped access. For example, a file must be 4MB or less and the total of all short-mapped access files for a process must be less than 6MB. Long-mapped access provides a 64-bit pointer to access a file of any size, up to the previous maximum file size of 4GB.Problem Description
Due to the way that existing MPE/iX compilers emit instructions to access memory, problems can result when accessing Large Files with a Large File pointer. Long pointers on PA-RISC hardware are defined to have two separate parts; a space ID, and an offset within that space. This also corresponds to the way virtual memory is defined in PA-RISC -- a 64-bit address broken down into the high 32-bits defining the space of the address, and the lower 32-bits defining the offset within that space. When building instructions to access memory, the space ID must first be loaded into a space register. The offset portion of the address is then loaded into a general register and the space register/general register pair is used to de-reference virtual memory. When adding to the pointer, the general register is incremented and then used in conjunction with the space register that was previously loaded. When the maximum file size supported on the system was 4GB (the size of each space), all valid file offsets were found within a single space, and there was no need to change the space ID. The advent of Large Files makes it possible for a single file to cross multiple space ID boundaries. This presents a problem for existing code which uses long-mapped access. When the general register holding the offset is incremented to the point that it overflows, logically it should reference the first word in the next space. However, because the space ID is held in a separate space register, the overflow of the general register is lost. The net effect is that access loops back to the beginning of the current space rather than moving on to the next space as intended.Compatibility Issue
In order to prevent applications from inadvertently stumbling across this data access problem, normal long-mapped access to Large Files has been made illegal. Any attempt to open a Large File for long-mapped access (HPFOPEN option 21) results in an error. In order to open a Large File for user-mapped access, a new large-mapped option must be passed (HPFOPEN option 87). This new large-mapped option works on both Large Files as well as normal-sized files. In programs that wish to continue to access Large Files through user-mapped access, all instances of HPFOPEN calls for long-mapped access (option 21) need to be changed to large-mapped access (option 87). The pointer that is returned is still a 64-bit pointer which is equivalent in every way to the pointer type returned by the long-mapped option. By specifying this new large-mapped access option, the option assumes that the caller is indicating their awareness of cross space ID problem and ensures that the program logic correctly handles any cross space ID buffer.Workaround
The workaround to this problem is actually fairly simple. Any access that would normally span a space ID boundary needs to be broken into two separate accesses, one for each side of the space ID boundary. This can be done within the calling application directly, or by calling the new intrinsics described in section 3 of this document. In order to open a Large File for user-mapped access, the new HOP_OPTION_LARGE_MAPPED (87) option must be passed instead of the long-mapped option.FLABELINFO
Two current items of the FLABELINFO intrinsic are unable to return information on some Large Files. These items are described in the following table:
| Item # | Description | New Item # |
Description |
|---|---|---|---|
| 28 | This item returns the size of the file limit of a file in bytes. It returns the value in an unsigned 32-bit integer. For any Large File, an error is returned if this item is called since the true file limit could not be expressed in a 32-bit value. | 62 | This item returns the number of bytes in the file limit, but instead returns the value in a signed 64-bit integer. This item can be used for all files regardless of whether or not they are Large Files. |
| 58 | This item was recently added to MPE/iX to return the number of sectors of disk space currently allocated to a file. This value is returned in a 32-bit signed integer. While this is adequate for all files supported as of 6.5, future release may support file sizes which will exceed the capacity of this item (512 GB). | 63 | This item forms a replacement for item 58. It also returns the number of sectors allocated to a file, but returns this information in a signed 64-bit integer. Because of this increased size, this item can reliably be used to get this disk space information for any sized file. |
32-bit Integer Overflow
There are a number of FFILEINFO, FGETINFO and FLABELINFO items that return information on the maximum file size of a file, or the current size of a file. While each of these items (with the exception noted above) works without change on Large Files, it is possible that programs that perform arithmetic on the values returned from these functions could experience integer overflow traps when dealing with very large files. An example of such an operation is multiplying the number of records in a file by the record size to get an estimation of the total size of a file. With normal-sized files, the result of this multiplication is guaranteed to never overflow an unsigned 32-bit integer. However, with Large Files, the result of this multiplication could be a value up to 128 GB, which could be much larger than a value that can be represented in 32-bits. In such cases, the result of the multiplication would need to be assigned to a 64-bit integer.
AIF:OS Impact
The Architected Interfaces for operating system information (AIF:OS) are a set of privileged routines that provide a supported means of accessing and altering internal system data structures. Because of the need to enhance these data structures to provide Large File support, some corresponding changes to a few AIF:OS routines are required. Each of the affected AIF:OS items follows a similar form. A 32-bit value is currently being returned, which cannot express the capacity of a Large File. In each case a new item number has been defined which works with a 64-bit value and can be used safely for either large or normal sized files. The following table describes each of the AIF:OS items that return errors for Large Files, as well as the new items that have been added to supplant the older items:
OPTIONS Enhancement
As a side effect of adding support for the KSAM64 file type, a new file type value, 7, has been allocated for KSAM64 files. The foptions structure is a 16-bit value made up of various bit fields. Several intrinsics accept or return foptions including FOPEN, FFILEINFO, FLABELINFO, and FGETINFO. The definitions of the foptions are:
The addition of KSAM64 support in no way alters the format of the foptions structure. Rather, a previously undefined value in the file type field, 7, is now defined. The list of defined file types which can be returned in the foptions is:
0:1 Reserved 1:1 Record format extension bit 2:3 File Type 5:1 File Equations disallowed 6:1 Labeled tape 7:1 Carriage control 8:2 Record format 10:3 Designator type 13:1 Binary/ASCII 14:2 File domain
In general, the definition of this previously undefined value will not cause any compatibility issues for programs. The only time that issues could arise is if code performed some type of validity check on the file type field of the foptions. In the case that user code is checking and rejecting a value of 7 in the file type field, the workaround would be to eliminate this check, or add 7 to the list of valid values.
0 Ordinary file type 1 CM KSAM 2 RIO 3 KSAMXL 4 CIR 5 SPOOL 6 MSG 7 KSAM64
Command Interpreter Changes
Although the impact of Large Files is negligible on most CI commands, there are several minor enhancements to a few commands. All CI commands work on Large Files in the exact same manner as they do for normal sized files. RENAME still renames Large Files, PURGE still purges them, and so on. The commands documented in this section have been enhanced to support some Large File features.
BUILD Command Changes
With the first release of Large File support, only two file types are being supported for Large File sizes; ordinary, fixed length record files, and KSAM files. Because of inherent limitations in the internal format of KSAMXL and CM KSAM files, it is necessary to introduce a new type of KSAM file, KSAM64 files, in order to go beyond a 4GB sized KSAM file. The internal format of the KSAM64 index area is structured so that file sizes from 1 record all the way to 1TB can be easily supported.While unfortunately, this means that MPE/iX supports three different varieties of KSAM files (CM KSAM, KSAMXL and KSAM64), the good news is that all three of these file types are functionally equivalent and can be used interchangeably by programs. The only external difference between KSAMXL and KSAM64 files is that a different file type must be used to create them. Other than this, all other KSAM options are specified identically. The only change to the BUILD command is the addition of the KSAM64 file type keyword. This keyword can be included in the BUILD command as a replacement for the KSAMXL keyword. KSAM64 files may be of any file size, but are the only KSAM file type that can be larger than 4GB. The new syntax for the BUILD command is:
BUILD filereference [F]
[;REC=[[recsize][,[blockfactor][,[U][,BINARY]]]]]
[V][,ASCII ]
[B]
[{;CCTL }]
{;NOCCTL}
[;TEMP]
[;DEV=[[dsdevice]#][device]]
[;CODE=filecode]
[;DISC=[numrec][,[numextents][,[initialloc]]]]
[{;RIO }]
{;NORIO}
{;STD }
{;MSG }
[{;CIR }]
{;KSAMXL}
{;SPOOL }
{;KSAM64}
[;ULABEL=numlabels]
[;KEY={^filereference2}]
{keyinfo }
[;FIRSTREC=recnum]
[{;REUSE }]
{;NOREUSE}
[;LANG={langid }]
{langname}
[{;DEFBLK }]
{;OPTMBLK}
In order to create a Large File, all that is required is to
specify a file limit of a sufficiently large number. For example,
to create a ordinary, fixed length record file with 80 byte records
that had a maximum capacity of about 8GB, a file limit of 100,000,000
records would need to be specified. For example, the following command
could be used:
BUILD TESTFILE;REC=-80,,F,ASCII;DISC=100000000Specifying a file limit that is greater than 4GB for any type of file other than ordinary fixed length record files and KSAM64 files will result in an error.
FILE Command Changes
The FILE and BUILD command syntaxes always mirror each other. It follows, then, that the new KSAM64 file type keyword is also being added to the FILE command, in the exact same way as the BUILD command. The new FILE syntax is included below:
=*formaldesignator
=$NULL
=$NEWPASS
FILE formaldesignator =$OLDPASS
=$STDIN
=$STDINX
=$STDLIST
=filereference[:nodespec]
[,filedomain]
[;DEV=[[envname]#][device][,outpri][,numcopies]
[;VTERM]
[;ENV=envfile[:nodespec]]
[;option]
[;access]
[;disposition]
[{;DEFBLK }]
{;OPTMBLK}
option Any valid option for the FILE command.
SYNTAX FOR OPTION
{F}
;REC=[recsize][,[blockfactor][,{U}[,{BINARY}]]]
{V} {ASCII }
;DEN=[density]
;DISC=[numrec][,numextents][,initialloc]
;CODE=[filecode]
[ {;RIO } ]
{;NORIO}
{;STD}
{;KSAMXL}
{;SPOOL}
{;KSAM64}
[ {;MSG} ]
{;CIR}
[;ULABEL=numlabels]
[;KEY={^filereference2}]
{keyinfo}
[;FIRSTREC=recnum]
[ {;REUSE} ]
{;NOREUSE}
[;LANG= { langid } ]
{ langname}
LISTF[ILE | TEMP] Command Changes
The LISTF, LISTFILE, and LISTFTEMP commands all can be used to display information about Large Files. All of the various LISTF and LISTFILE formats continue to work with normal sized files as well as Large Files. File type information about KSAM64 files is displayed in the LISTF family of commands as a lowercase 'k'. In order to display information about Large Files, two new file format options have been introduced. The LISTF, LISTFILE, and LISTFTEMP commands now support options 10 (SUMMARYWIDE) and 11 (DISCWIDE). File format option 10 is the Large File equivalent of file option "1", containing all the same information, but in a new format that allows for the expression of larger values. Similarly, the format option 11 is the Large File equivalent of file option "2". Again, the same basic information is displayed, but the format has been changed to allow for a greater range of values to be displayed. The syntax of the LISTF, LISTFILE, and LISTFTEMP commands has not been changed with the addition of the new 10 and 11 format options. The same syntax can be used with the new values of 10 and 11 as the format. The output of the new formats varies depending on whether or not MPE/iX syntax names are being displayed or HFS syntax names are being displayed. The format for each of the MPE/iX and HFS syntax varieties of the new formats 10 and 11 can be seen from the examples shown below:
:listf @,10
ACCOUNT= SYS GROUP= EXAMPLE
Name Access FCode RecSiz Type EOF File Limit
-------- ERWS ----- ------ ----- ------------- -------------
BIGFILE 1024 FA 0 1023456789
ICE E NMPRG 256 FB 832 832
XKSM64 80 FAk 1 1023
XKSMXL 80 FAK 1 1023
XRAND RW 80 FA 157 157
XRAND2 W 252 VA 396 49
YRAND NMOBJ 256 FB 22 4000
:listfile ./@,10
PATH= /SYS/EXAMPLE/
Access FCode RecSiz Type EOF File Limit Name
ERWS ----- ------ ----- ------------- ------------- ------------
1024 FA 0 1023456789 BIGFILE
E NMPRG 256 FB 832 832 ICE
80 FAk 1 1023 XKSM64
80 FAK 1 1023 XKSMXL
RW 80 FA 157 157 XRAND
W 252 VA 396 49 XRAND2
NMOBJ 256 FB 22 4000 YRAND
:listf @,11
ACCOUNT= SYS GROUP= EXAMPLE
Name Access FCode RecSiz Type EOF File Limit Disk Usage Exts
-------- ERWS ----- ------ ---- ------ ---------- --------KB -----
BIGFILE 1024 FA 0 1023456789 0 0
ICE E NMPRG 256 FB 832 832 208 1
XKSM64 80 FAk 1 1023 64 1
XKSMXL 80 FAK 1 1023 52 1
XRAND RW 80 FA 157 157 16 1
XRAND2 W 252 VA 396 49 16 1
YRAND NMOBJ 256 FB 22 4000 8 1
:listfile ./@,11
PATH= /SYS/EXAMPLE/
Access FCode RecSiz Type EOF File Limit Disk Usage Exts Name
ERWS ----- ------ ---- ------ ---------- --------KB ----- --------
1024 FA 0 1023456789 0 0 BIGFILE
E NMPRG 256 FB 832 832 208 1 ICE
80 FAk 1 1023 64 1 XKSM64
80 FAK 1 1023 52 1 XKSMXL
RW 80 FA 157 157 16 1 XRAND
W 252 VA 396 49 16 1 XRAND2
NMOBJ 256 FB 22 4000 8 1 YRAND
The display for the format options 10 and 11 is the same for each of the
LISTF, LISTFILE, and LISTFTEMP commands. One of the unique
additions of these two new options is the "ERWS" column. This column
indicates whether or not the file is currently opened, and if so,
for what type of access. An "E" in the E column
indicates that the file is opened for Exclusive access, the "R" in
the R column indicates that the file is opened for Read access, the "W" in
the W column indicates that the file is opened for Write access,
and the "S" in the S column indicates that the
file is being Stored by the STORE or TurboSTORE utility. The
column may have one or more of these characters specified, or none, depending
on how the file is currently accessed.
Intrinsics Changes
As part of the addition of Large File support to MPE/iX several intrinsics changes have been introduced. This encompasses both the modification of existing intrinsics, as well as the introduction of a new set of intrinsics. This chapter describes all of the intrinsics changes that have been implemented as a part of Large File support.
Existing Intrinsics
Relatively few changes have been necessary to the existing file system intrinsics in order to implement Large Files. The majority of these changes are in support of adding 64-bit return values for file byte offsets.HPFOPEN Changes
As described in the section "User-Mapped Access to Files", when performing user-mapped access to Large Files, care must be taken to not attempt to dereference a buffer that crosses a SID boundary. In order to ensure that programs are aware of this caveat, traditional attempts to open a Large File for user-mapped access will fail. In order to open a Large File for user-mapped access, a new HOP_OPTION_LARGE_MAPPED option (item number 87) must be used, as described in the following table:| Item Name | Item Number | Item Size | Description |
|---|---|---|---|
| HOP_OPTION_LARGE_MAPPED | 87 | @64 | This item returns a 64-bit pointer value which points to the start of the file data of the specified file. This item works for any sized file, but is the only user-mapped access option that works for Large Files (files whose file limit is greater than 4GB). Note that not all file types allow user-mapped access. If this item is requested against a file type that does not support user-mapped access, the HPFOPEN request will result in a failure. |
FLABELINFO Changes
Three new items will be supported by the FLABELINFO intrinsic. These new options are described in the table below:| ItemName | Item Number | Item Size | Description |
|---|---|---|---|
| FLI_FILE_LIMIT_BYTES_64 | 62 | 8 bytes | This item returns a 64-bit integer which indicates the maximum file size of the specified file in bytes. This is the 64-bit equivalent of FLABELINFO item 28. |
| FLI_COMPUTED_SECTORS_64 | 63 | 8 bytes | This item returns a 64-bit integer which holds the number of 256 byte sectors of disk space that are currently allocated to the specified file. This item is the 64-bit equivalent to the FLABELINFO item 58. |
| FLI_FILE_LIMIT_IS_LARGE | 64 | 4 bytes | This item returns a 32-bit integer which can be used as a simple test to see if the file limit of a file is greater than 4GB. This item will return two values:0 = The file limit is less than 4GB1 = The file limit is greater than 4GB |
FFILEINFO Changes
Two new items have been added to the FFILEINFO intrinsic. These items are described in the table below:| Item Name | Item Number | Item Size | Description |
|---|---|---|---|
| FF_FILE_LIMIT_BYTES | 109 | 8 bytes | This item returns a 64-bit integer indicating the maximum size of the file, or the file limit, in bytes. |
| FF_FILE_LIMIT_IS_LARGE | 110 | 4 bytes | This item returns a 32-bit integer which can be used to easily tell if
the specified file has a file limit greater than 4GB. This item returns two
values:0 = The file limit is less than or equal to 4GB 1 = The file limit is greater than 4GB. |
New Intrinsics
The following new intrinsics are being added to aid in the support of user-mapped file access. These intrinsics can be used for pointer manipulation and data movement on any user-mapped file; short-mapped, long-mapped, or large-mapped. They are especially recommended for use with the large-mapped HPFOPEN option.
HPFADDTOPOINTER
NM callable only.This routine can be used to perform arithmetic on a 64-bit pointer value. Byte offsets can be added to or subtracted from a pointer by specifying either a positive or negative offset value. Syntax
@64 I64 @64 I32
HPFADDTOPOINTER (base_ptr, offset, return_ptr, status);
Parameters
- base_ptr
64-bit pointer by reference (required)
The base_ptr can be a 64-bit pointer to an object of any type.- offset
64-bit signed integer by reference (required)
The offset can be any positive or negative value. Specifying a positive value moves the return_ptr forward from the previous base_ptr, while a negative value moves the return_ptr backward from the base_ptr.- return_ptr
64-bit pointer by reference (required)
The return_ptr is an output parameter that has the new pointer value returned to it. It can be a 64-bit pointer to an object of any type.- status
32-bit signed integer by reference (optional)
Returns the status of the HPFADDTOPOINTER call. If no errors or warnings are encountered, status returns 32 bits of zero. If errors or warnings are encountered, status is interpreted as two 16-bit fields. Bits (0:16) comprise status.info. A negative value indicates an error condition, and a positive value indicates a warning condition. Bits (16:16) comprise status.subsys. The value represents the subsystem that set the status information.
 |
NOTE: No attempt is made to verify that the pointer value returned is a legitimate pointer to a valid object. Any invalid pointers are detected and generate errors when the pointers are dereferenced. |
HPFMOVEDATA
NM callable only. This routine can be used to efficiently move data from a source buffer to a target buffer. Syntax
I64 @64 @64 I32
HPFMOVEDATA (count, source_ptr, target_ptr, status)
Parameters
- count
64-bit signed integer by reference (required)
The count parameter allows the caller to specify the number of bytes to move from the source buffer to the target buffer.- source_ptr
64-bit pointer by value (required)
The source_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- target_ptr
64-bit pointer by value (required)
The target_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- status
32-bit signed integer by reference (optional)
Returns the status of the HPFMOVEDATA call. If no errors or warnings are encountered, status returns 32 bits of zero. If errors or warnings are encountered, status is interpreted as two 16-bit fields. Bits (0:16) comprise status.info. A negative value indicates an error condition, and a positive value indicates a warning condition. Bits (16:16) comprise status.subsys. The value represents the subsystem that set the status information.
|
NOTE: When calling the HPFMOVEDATA intrinsic, it is important to ensure that the source and target buffers are not overlapping. The results of a HPFMOVEDATA call when source and target buffers are overlapping are undefined. If source and target buffers are overlapping, the HPFMOVEDATALTOR or HPFMOVEDATARTOL intrinsics should be used. |
HPFMOVEDATALTOR
NM callable only. This routine can be used to efficiently move data from a source buffer to a target buffer. If the source and target buffers were viewed horizontally, like a line of text, the data movement is performed by starting at leftmost position of the source buffer (to the leftmost position of the target buffer) and proceeding to the rightmost. Syntax
I64 @64 @64 I32
HPFMOVEDATALTOR (count, source_ptr, target_ptr, status)
Parameters
- count
64-bit signed integer by reference (required)
The count parameter allows the caller to specify the number of bytes to move from the source buffer to the target buffer.- source_ptr
64-bit pointer by value (required)
The source_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- target_ptr
64-bit pointer by value (required)
The target_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- status
32-bit signed integer by reference (optional)
Returns the status of the HPFMOVEDATALTOR call. If no errors or warnings are encountered, status returns 32 bits of zero. If errors or warnings are encountered, status is interpreted as two 16-bit fields. Bits (0:16) comprise status.info. A negative value indicates an error condition, and a positive value indicates a warning condition. Bits (16:16) comprise status.subsys. The value represents the subsystem that set the status information.
|
NOTE: This intrinsic is especially useful when the source and target buffers are overlapping. The HPFMOVEDATALTOR intrinsic is typically used when the target buffer's address is to the left (smaller) of the source buffer's address. Moving the data from the left to the right ensures that the data in the source buffer is copied to the target buffer before it is overwritten itself. |
HPFMOVEDATARTOL
NM callable only. This routine can be used to efficiently move data from a source buffer to a target buffer. If the source and target buffers were viewed horizontally, like a line of text, the data movement is performed by starting at the rightmost position of the source buffer (to the rightmost position of the target buffer) and proceeding to the leftmost. Syntax
I64 @64 @64 I32
HPFMOVEDATARTOL (count, source_ptr, target_ptr, status)
Parameters
- count
64-bit signed integer by reference (required)
The count parameter allows the caller to specify the number of bytes to move from the source buffer to the target buffer.- source_ptr
64-bit pointer by value (required)
The source_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- target_ptr
64-bit pointer by value (required)
The target_ptr can be a 64-bit pointer to any valid object that the calling process has access to. The buffer may be in the caller's stack, heap, or obtained by opening a file with user-mapped access.- status
32-bit signed integer by reference (optional)
Returns the status of the HPFMOVEDATALTOR call. If no errors or warnings are encountered, status returns 32 bits of zero. If errors or warnings are encountered, status is interpreted as two 16-bit fields. Bits (0:16) comprise status.info. A negative value indicates an error condition, and a positive value indicates a warning condition. Bits (16:16) comprise status.subsys. The value represents the subsystem that set the status information.
|
NOTE: This intrinsic is especially useful when the source and target buffers are overlapping. The HPFMOVEDATARTOL intrinsic is typically used when the target buffer's address is to the right (larger) of the source buffer's address. Moving the data from the right to the left ensures that the data in the source buffer is copied to the target buffer before it is overwritten itself. |
HPFFILLDATA
NM callable only. This routine can be used to efficiently initialize a buffer with a specified character value. Syntax
I64 @64 CV I32
HPFFILLDATA (count, buffer_ptr, fill_char, status)
Parameters
- count
64-bit signed integer by reference (required)
A positive count of the number of bytes in the buffer indicated by the buffer_ptr parameter that should be initialized.- buffer_ptr
64-bit pointer by value (required)
A pointer to the buffer that should be initialized. The buffer_ptr may point to any valid object in your stack, heap, or a file that has been opened with user-mapped access.- fill_char
Character value by value (required)
The character value that should be used to initialize the specified buffer. Any value in the range of 0 through 255 can be specified, including all printable and non-printable ASCII characters.- status
32-bit signed integer by reference (optional)
Returns the status of the HPFFILLDATA call. If no errors or warnings are encountered, status returns 32 bits of zero. If errors or warnings are encountered, status is interpreted as two 16-bit fields. Bits (0:16) comprise status.info. A negative value indicates an error condition, and a positive value indicates a warning condition. Bits (16:16) comprise status.subsys. The value represents the subsystem that set the status information.
Miscellaneous
This section covers several issues that are related to Large Files that may be of interest to the reader.
STORE/RESTORE Impact
The STORE and RESTORE utilities have been modified to handle Large Files. This enhancement was accomplished in such a fashion that STORE tapes with Large Files are completely compatible with previous versions of the operating system. A STORE tape created on release 6.5 can be taken to any other MPE/iX system and be successfully restored. On systems prior to 6.5, any Large File on the tape is not recognized as a valid file and will not be restored onto the system. Although Large Files are skipped over in this scheme, all other files will be able to be restored. Any attempt to store a Large File using the ;TRANSPORT STORE option results in an error at store time. In addition to the STORE/RESTORE and TurboSTORE products all supporting Large Files, most, if not all, 3rd party backup products have also been enhanced to support Large Files. If you rely on 3rd party backup tools, please refer to your vendor for information regarding the proper updates and procedures for using these tools on release 6.5.Backdating Procedures
If for some reason it is necessary to "backdate" your system from the 6.5 release to a prior MPE/iX release, very little extra work is required due to Large Files. Before attempting the backdate operation, the 6.5 system must be shut down completely with the CNTRL-A SHUTDOWN command. You must also wait for the final system shutdown message to appear before resetting the system and beginning the UPDATE command on the earlier MPE/iX release. Any Large Files that are in the directory structure of the system are unusable once the system is backdated to a release earlier than 6.5. The Large Files can be purged from the older system via the :PURGELINK command. Any other attempt to open or manipulate a Large File results in an error. Once the system is again updated to release 6.5 or beyond, the Large Files are once again accessible.Utility and Subsystem Support
Nearly all utilities and subsystems support Large Files without change. This runs the spectrum from FCOPY to SORT. The only word of caution is that not all utilities and subsystems support file sizes that are less than 4GB. This includes such things as EDITOR or HPEDIT. The general rule of thumb is if it doesn't work on a 3GB file, it won't work on a Large File. One exception of note is the DSCOPY subsystem. DSCOPY has not been enhanced to be able to transfer Large Files, or the new KSAM64 files. The FTP utility has been enhanced to do network transfers of these files, and is the preferred utility for network file transfer.Impact on IMAGE Database
The introduction of Large Files has had no impact on the IMAGE/SQL product. Large database support is still accomplished by use of Jumbo Datasets. Therefore there are no compatibility issues to be concerned with in regards to changes in your IMAGE/SQL databases.HP SRC
Version A.01.29 of HP SRC does not support KSAM64 files. If an attempt is made to check in a KSAM64 file, error 387 will be reported.JAVA
Java programs will be able to access large MPE/iX files by using Java Native Interface (JNI) to call MPE/iX intrinsics, but will not be able to access Large Files via the java.io APIs. The java.io APIs treat all files as bytestream files and access these files through the POSIX C language APIs. The first release of Large Files for MPE/iX does not include support for bytestream files larger than 2GB in size; therefore, there can be no Large Files that are also bytestream files in this release.HPSEARCH/iX
HPSearch/iX has been changed to provide for searching of Large Files. Additionally, it has been enhanced to search KSAM64 and KSAMXL files.HPGLANCEPLUS/iX
HPGlancePlus/iX has been changed to display file information about Large Files. Also, KSAM64 files are included in the File Display on the Process Screen and will show a type of KSM64.HPBROWSE/iX
HPBrowse/iX supports viewing Large Files and KSAM64 Files.HPEDIT/iX
HPEdit/iX does not support files of 2,000,000 records or more, and so, does not support Large Files.COBOL/iX
COBOL II/iX supports Large Files. No changes are necessary to existing COBOL II/iX programs to read, write, and update Large Files. KSAM64 files are supported by the Indexed I-O module. When you open a file with indexed organization, that file is permitted to be an existing KSAM64 file. When COBOL II/iX creates a file with indexed organization, by default it will create a KSAM XL file. However, you may override the default by issuing a FILE command specifying KSAM64 as the file type.COBOL II/CM
COBOL II/CM does not support KSAM64 files. An attempt to open or create a KSAM64 file with a CM COBOL program will result in file status 39 and, typically, a program abort with COBOL error 648.C/iX
C/iX does not support Large Files. You must issue your own intrinsic calls if you want to access Large Files from C. Attempting to open a Large File using the C/iX Library function fopen() will fail and set errno to ESYSERR (50).|
Chapter 3 Growth Solution |
AIFs for Large Files |