Checkpoints

A checkpoint is a snapshot of the state of a printer at a particular point in the data stream. If that point happens to be at the top of page N, and the native mode spooler can restore that state to the printer, it starts printing immediately at page N without having to silent run through pages 0 to N-1.

Checkpoints are a feature of the CIPER protocol, implemented only in the HP-IB version of HP256x printers. The HP-IB printers generate checkpoints at the top of each page.

How do checkpoints work?

The spooler sends data through the file system to the printer, and the printer begins to print it. At some point, the printer advances paper to the top of the second page. It generates a checkpoint, and notifies the spooler (through many layers of status reporting) that the checkpoint is available for retrieval. Whenever the spooler receives this status, it requests the checkpoint from the printer and saves it in the checkpoint file.




	NOTE: Because of buffering in the data path, as well as the asynchronous nature of checkpoint reporting, the spooler may be well into data destined for page 2 (or even page 3 or later) of the report by the time it receives the checkpoint for the end of page 1.

Printing continues. Suppose at page 100 an operator enters SPOOLER <ldev>; SUSPEND; OFFSET=-20. Since the ;NOW keyword is assumed by default, the spooler flushes all buffered data to the printer, sets the target page to 80, and suspends.

Later the operator enters SPOOLER <ldev>; RESUME. Using the recovery process described earlier, it fetches the target page (80), searches the checkpoint file for the corresponding checkpoint, downloads the checkpoint to the printer, then begins sending data from (near) that point in the spool file. The printer silent runs for a short period until the proper data for page 80 arrives, then it begins to print. Note that it was not necessary for the spooler to send the first 79 pages.

Checkpoint files

Although only CIPER protocol devices return checkpoints, the spooler always creates a checkpoint file. This is primarily for device independent operation. The checkpoint file always has a dummy "checkpoint 0" entry. When silent running is invoked, the spooler first looks for the closest checkpoint preceding the target page as a starting point for silent running. The checkpoint 0 entry assures that such a checkpoint always exists. If it is used, the silent run starting point is the beginning of the spool file, as expected.

The spooler creates a unique checkpoint file for each device which prints a given spool file for the following reasons:

Different devices result in different files. As noted earlier, CIPER protocol devices return checkpoints, other devices do not. A checkpoint file generated as a result of printing to a CIPER protocol device would not be usable if another copy was printed to an HP 2680.
Even among similar devices, the state information in a checkpoint may vary. If an existing checkpoint file entry does not exactly match that of the entry for the same page returned from the printer, the spooler marks the checkpoint file as corrupt. This is discussed in more detail in the "Reality check" section later in this appendix.

Usable for additional copies

The spooler generates a checkpoint file as it prints the first copy of a spool file. Once generated, the file can be used when printing subsequent copies. A later example in the "No checkpoints while silent running" section, shows how a forward search (page 100 in the example) requirs a lengthy silent run cycle. If, instead, the first copy had printed without interruption, it would be possible to start a second or subsequent copy (on the same printer) at page 100 with a minimal silent run delay. This is because the checkpoint for page 100 was generated as the first copy was printed, and can now be used to go directly to page 100.

A checkpoint file is deleted with its spool file

Because the checkpoint file is created automatically by a spooler process, it is also deleted automatically when the corresponding spool file is deleted, either following its final copy, or when deleted by a command (SPOOLF or PURGE). The checkpoint file is also deleted if the spool file moves to the SPSAVE state.

If more than one checkpoint file exists for the same spool file, because the spool file was printed on more than one device, they are all deleted.

Occasionally a checkpoint file remains after its spool file has been deleted. Prior to MPE/iX Release C.50.00, this usually occurred because a spool file was being stored to a tape when a spooler process tried to delete it. The Store process deleted neither the spool file nor the checkpoint file when it finished storing them. This problem has been corrected in MPE/iX Release C.50.00.

Any user with SM capability can purge any checkpoint file which is not being accessed. Even if the corresponding spool file still exists, no harm is done. If the file is selected for printing, the spooler generates a new checkpoint file. If the file is deleted without being printed, the system does not care if no checkpoint file exists.

File naming convention

The spooler generates one checkpoint file per spool file for each device which prints any part of the spool file. A checkpoint file is named Cnnnnn, where nnnnn is the numerical value of the spoolid of the corresponding spool file. The checkpoint filename is placed in device name groups in the HPSPOOL account.

There are two ways to name a checkpoint file:

By the corresponding spool file and the device name
For example, suppose LP1 is a device name, and the native mode spooler creates the output spool file O1234.OUT.HPSPOOL that is printed by device LP1. Then the name of the companion checkpoint file is C1234.LP1.HPSPOOL.
By the spool file and the logical device number
Whenever a spooler process is started for a device, a group is created in the HPSPOOL account (if it does not already exist). This group is named Dmmmmmmm, where the mmmmmmm represents the logical device number (with leading zeros as required) of the device.
For example, if a spooler process is started for LDEV 6, the corresponding group in HPSPOOL is D0000006. If #O1234 is selected for printing by LDEV 6's spooler process, the spooler creates checkpoint file C1234.D0000006.HPSPOOL (unless it already exists from an earlier selection by the same spooler process). If #O1234 is later routed to LDEV 19, LDEV 19's spooler creates C1234.D0000019.HPSPOOL if necessary.

Checkpoint file space

Checkpoints are not new with the native mode spooler. They were introduced on MPE V/E when the HP-IB CIPER printers were first released. Their generation is strictly a printer phenomenon, but the spooler catalogs them and uses them in a recovery situation.

On MPE V/E, the catalog of checkpoints was in the spool file's user labels. The number of user labels available, and the size of the checkpoint structure limited the number of checkpoints to 32. These were managed in a circular fashion, with the 33rd checkpoint replacing the first, and so on. The spooler stored every eighth checkpoint (remember, the printer generates one at the top of each page), allowing a range of 256 pages. This is why the RESUMESPOOL command had a 256 page limitation, and why successive iterations of the command to exceed 256 pages did not work.

The native mode spooler removed the arbitrary 256 page limitation by cataloging checkpoints in a separate file. This file theoretically can grow to the maximum 4Gb, but that would never happen, as any reasonable spool file contains much more data per page than a checkpoint entry. Therefore, a hypothetically very large spool file would hit the 4Gb limit long before its checkpoint file did.

In practice, the initial allocation of checkpoint file space is quite small. In addition, non-CIPER printers do not generate checkpoints. Therefore, the file does not grow beyond its initial allocation, so disk space for checkpoint files is usually not a problem. Even though checkpoint files are used for non-CIPER printers (because they are managed by the spooler, which must be device independent), such files contain only a few structural entries.

Corrupted checkpoint files

As mentioned earlier, whenever the spooler for a particular CIPER protocol device prints the same page more than once, there exists a cataloged checkpoint in the checkpoint file, and a checkpoint which is returned by the device. The spooler enforces an exact match between these two. If it detects any difference, the checkpoint file is marked corrupted. Subsequent checkpoints are not cataloged, nor are any earlier ones reused. Note that any recovery situation then requires silent running from the beginning of the spool file.

The only way to re-establish the use of the checkpoint file is to start a new copy of the spool file at the beginning. This logically deletes any earlier contents and marks the checkpoint file as usable. Any checkpoints received during this copy are then cataloged normally.

Why are such stringent conditions enforced? Because the same spool file printed on the same device should yield the same checkpoints. If it does not, the checkpoint file data is suspect and should not be used. The only disadvantage is that any recovery situation requires more time than if a valid checkpoint file were available. But this approach also assures that no error occurs as a result. The recovery will succeed, it just takes longer.

Checkpoints versus silent running

Note the difference between checkpoints and silent running. The HP 2680, HP 5000, color LaserJet, LaserJet 4 family, and all CIPER printers (HP-IB versions of HP 256x printers) all support silent running. Only the CIPER printers support checkpoints. Checkpoints make it possible to minimize the time a printer spends silent running.

Serially-connected printers do not have hardware support of silent running. The serial printer storage manager emulates silent running for a subset of possible conditions. Recall that full emulation would require a complete device-specific print engine to account for all possible vertical motion control sequences.

For the purpose of supporting silent running, the serial PSM assumes 66 lines per physical page, with top and bottom margins of three lines each. It also uses the HP standard default VFC specification. A user-specified VFC (using FDEVICECONTROL 64) is not supported. Within those limitations, it recognizes and interprets the full range of carriage control codes, in both prespace and postspace modes. However, it does not scan data looking for PCL or other vertical motion sequences.

Checkpoint considerations

Must be CIPER printer

Only CIPER protocol printers support checkpoints. The only CIPER printers are the HP 256x series with HP-IB interface. Why do other printers not provide checkpoints? Because checkpoints are a snapshot of a printer state, and the large page printers such as the HP 2680 and HP 5000 simply have too much state information. The font table (and all the downloaded bitmapped fonts), the forms overlay table, the logical page table are all part of the printer state. It would require an extremely large checkpoint file to save all this information for each page printed. It would also take substantial link bandwidth to upload the data from the printer, thereby subtracting from overall printer performance.

The HP 2680, HP 5000, and Color LaserJet, and LaserJet 4 family printers do not return checkpoints, but do support silent run. This means that in a recovery situation it is not necessary to reprint the entire file, but it is necessary for the spooler to transmit the file from its beginning (with the printer in silent run mode until the target page is reached). In our earlier example, this corresponds to silent running through the first 79 pages of data instead of going directly to page 80. The reason for this is that the beginning of the file is the only place where the spooler and the printer agree on the printer's state. Without checkpoints, there is no way for the spooler to know the printer's state as of a given page. Therefore, there is no way to restore that state in a recovery situation.

Non-recoverable checkpoints

Even with a CIPER printer, a checkpoint is only valid when it represents the full state of the printer as of a given page. Such a checkpoint is a recoverable checkpoint, but "recoverable" is usually omitted from the description. If the state of the printer cannot be fully represented in the space available in the checkpoint structure, the checkpoint is called a nonrecoverable checkpoint, meaning it cannot be used in a recovery situation.

Downloading a Vertical Forms Control (VFC) image causes all succeeding checkpoints to be nonrecoverable. It is the only user option that causes nonrecoverable checkpoints. Since VFC images are usually downloaded at the beginning of a spool file, this means that the printer is forced to silent run from the beginning of the spool file in any recovery situation.

No checkpoints while silent running

The printer does not generate checkpoints when it is in silent run mode. This can be significant, as the following example shows:

Suppose the printer prints the first 20 pages of the first copy of a spool file. An operator then suspends and resumes the spooler at page 100. Because there is no checkpoint for page 100, the printer must silent run from page 21 to page 100, then resume printing. It does so, and at page 120 the operator again suspends spooling and resumes it at page 80. Because the printer did not generate checkpoints while it was silent running through pages 21-99, the closest checkpoint not exceeding page 80 is that for page 20. It is downloaded, but the printer must then silent run from there until page 80.

Checkpoint file must be logically consistent

Whenever the spooler retrieves a checkpoint from the printer, it checks whether or not that checkpoint has already been cataloged in the checkpoint file. If so, the cataloged checkpoint must identically match the one returned by the printer. If it does not, the checkpoint file is considered corrupted. The spooler does not use it unless a new copy of the spool file is started from the beginning, thereby regenerating the entire checkpoint file.

This phenomenon can lead to long silent running, even for spool files that previously generated recoverable checkpoints. Without a checkpoint close to the target page, the spooler is forced into silent running from the beginning of the spool file.

Checkpoints defined by printer's physical top-of-form

As mentioned earlier, CIPER printers generate a checkpoint at the top of each page. But there are two kinds of pages. The physical page length is settable only on the printer's front panel (that is, it cannot be set programmatically, and so is unavailable even to the spooler). Logical page lengths can be set through the proper PCL command sequence, but are usually set by the VFC definition in the CIPER ENV file specification. For example, VFC,6,66 defines a logical page length of eleven inches (66 lines at 6 lines per inch).

The printer generates a checkpoint each time it reaches the top of a physical page. If the physical page length is different from the logical page length, you do not get expected results in a recovery situation.

Printer must stop at physical top-of-form

For the printer to generate a checkpoint, paper must stop on the line representing the physical top-of-form. Many applications position paper in the most efficient manner by advancing from the final printed line on one form to the first printed line on the next. If this first line is not at the top of the physical form, the printer does not generate a checkpoint.