⇓   ⇐
  ←Part 22  Part 24→

The HP 3000--For Complete Novices
Part 23: IMAGE/SQL Application Design

Feature by George Stachnik
In the last few installments, we've learned how to perform a few simple but practical tasks using HP's IMAGE/SQL database management system. We've created an IMAGE/SQL database, and seen how to begin accessing it using QUERY. I have tried, in this series, to take a very practical approach to the HP 3000, and to IMAGE/SQL in particular. Rather than spending a lot of time on theory, I've tried to focus on how to get the job done, and to talk about theory only in that context. This time around, we're going to step out of that pattern and look at the design of IMAGE/SQL applications.

We will define an application to be a collection of computer programs and related components (documentation, hardware, etc.) that all work together to perform a particular task. One of the most important aspects of designing an application is figuring out just exactly how the programs are going to work together.

Way back in the 1970s, when the HP 3000 was introduced to the market, most applications were designed to run as batch jobs. Batch applications processed information in much the same way as an assembly line processed parts. Consequently, designing batch applications has a lot in common with designing an assembly line.

A batch application was made up of a collection of programs, each of which accepted data from one or more input files. Each program transformed the data somehow (sorted it, combined it, separated it) and produced one or more output files, which were in turn used as input files for the next program in the assembly line. Eventually, at the end of the line, the application produced something that people could use, typically a report of some kind. Batch applications also typically stored some information permanently. This permanent repository came to be known as a master file.

The HP 3000 came along at a time when database technology was transforming the way computers processed information. Business people were not happy with the constraints placed upon them by the batch assembly line way of processing information. Batch applications forced them to hand transactions over to a computer operator who would collect them together into "batches" (hence the name "batch job"). The batch would then be fed to the assembly line. Depending on how often this was done, users would have to wait for days or even weeks for the results of their transactions to come back to them.

Not surprisingly, this was not what most users had in mind. They wanted to be able to enter their transactions into the computer themselves and get the results back immediately. Thus, the idea of an online application was born.

The most popular platform for the creation of batch applications had been IBM's mainframes. When the online revolution began to roll over the information processing industry, IBM tried to adapt its mainframe platform to simultaneously support both batch and online environments. They were reasonably successful in doing this, but at a cost. Mainframes may have been good at processing batch jobs, but they made very inefficient and expensive online machines. Many computer vendors (notably HP) saw this as an opportunity to take some market share away from IBM. They figured out that online applications could be designed and run much more efficiently (that is, cheaply) on a machine that was designed and built specifically for that purpose. Thus the HP 3000 was designed and built primarily as an online machine that could also be used as a batch processor. Unlike the IBM mainframe, which was a fabulously cost-effective batch processor but an expensive and clunky online platform, the HP platform was an amazingly cheap and efficient online machine and a fair-to-middling batch platform.

HP determined that many of the technologies that had been used to create batch applications (for example, languages such as COBOL) could be used just as well for the design and creation of new online applications. Hence, COBOL (which had been the language of choice in the batch world) quickly became the lingua franca of the HP 3000 online world as well.

But there were a number of key differences in the technologies that were required to create batch and online applications. The first, and most obvious, was the need for terminals. HP's MPE operating system used an ingenious system of device independence, which treated terminals as if they were files, thus simplifying the programming required to interface these new devices to COBOL applications. The second, and far more revolutionary requirement, was the need for a database management system, or DBMS.

The Birth of Database Management Systems

We've seen that batch applications stored data in an ordinary file called a master file. Architecturally speaking, a master file wasn't anything special. A batch application's master file was little more than a sorted collection of records that could be accessed sequentially or directly using a record number. In some cases, the master file might be accessed using a key--which led to the development of KSAM. But the most important thing to understand about master files and batch applications is that the master file was never accessed by more than one program at a time. Batch application programs ran sequentially--one at a time--assembly-line fashion.

Online applications changed all that. With the advent of online applications, designers suddenly had to deal with the fact that an online application's data would be accessed not by one program at a time, but by multiple programs, each of which operated on the data in different ways. To complicate things further, each program could be used by multiple users at the same time.

At the time, this represented an apparently insoluble problem for application designers. To designers of traditional batch applications, the online world represented nothing less than chaos. Instead of a well-ordered assembly line, an online world seemed more like a football game--with gangs of unruly users all trying to get the ball (i.e., the data) at the same time.

Clearly, the software technologies that had been used to create batch assembly lines were not up to solving what soon became known as "the football problem": How do you control access to the data so that users can do what they want to do when they want to do it, and at the same time protect the data so that it doesn't become corrupted? How do you plan for the scenario in which two or more users need to update a particular file (or a particular record within a file) at the same time? Experiments soon showed that traditional batch technologies would either corrupt the file or force each user to wait so long that no user would use the system.

Hence the database management system (DBMS) was born. By the mid 1970s, most computer vendors, (IBM, DEC, HP) had created DBMS products and were offering them for sale. But HP leapfrogged its competition with a bold move. When it brought its HP 3000 business minicomputer to the market, it bundled the IMAGE DBMS with the system. Other vendors were still treating database technology as an add-on--something that their more adventurous customers would have to pay for. HP wisely figured out that most HP 3000 applications were going to need database technology if they were going to be successful. So instead of trying to gouge customers out of a few extra bucks for it, they effectively gave it away with the 3000.

The results can still be seen today. As the IMAGE DBMS has evolved (changing its name, first to TurboIMAGE, then to IMAGE/SQL), virtually every HP 3000 application continued to use IMAGE database technology. It's important to understand that IMAGE's broad acceptance in the HP 3000 community was not due only to the fact that the price was right (that is, free). IMAGE won awards in its day because of the unique and simple solutions it provided to the fundamental problems facing application programmers in the 1970s. Most notably, IMAGE provided a neat and simple solution to the football problem that is still applicable in today's world.

Solving The Football Problem

The first step in solving the so-called "Football problem" was to divide users up into classes. We've already seen how IMAGE does this in the last three articles in this series. We saw that each dataset and each data-item in an IMAGE database is associated with a list of class numbers. Each class number represents a certain class of users.

An individual user may have access to a particular database. But he may or may not have access to all the records (data entries) in the database, or to all the fields (data-items) in the records. Figure 1 shows the class numbers in the sample database schema that we've been using in these articles. They are defined in the PASSWORDS section, and referenced in the ITEMS section and the SETS section.

Contrast this with the way batch applications work. With the ordinary files used in batch systems, an individual user either has access to a file or he doesn't. If a user does have access to a file, he has access to all the data in that file.

Figure 1: Class Numbers 
BEGIN DATA BASE ORDERS;  << CUSTOMER ORDERS >> 



PASSWORDS: 

4 CLERK;        << SALES CLERK >> 

8 SUPER;        << SUPERVISOR OF CUSTOMER ORDERS DEPARTMENT >> 

15 DO-ALL;      << PROGRAMMER/ANALYST - CREATOR OF DATA BASE>> 

 

ITEMS:            << (READ / WRITE) >> 

ACCOUNT,        X6   (4,8,15/8,15);   << CUSTOMER ACCT=AA-NNN >> 
CITY,           X14  (4,8,15/8,15);   << CUSTOMER CITY        >> 
.
.
.
ZIPCODE,        X6   (4,8,15/8,15);   << CUSTOMER ZIPCODE     >> 
 
SETS: 

NAME:             CUSTOMER,MANUAL(4,8,15/8,15); 

ENTRY:            ACCOUNT(1), 
.
.
.

CAPACITY:         1231; 


END.
IMAGE's Database Access Modes

The second step in solving the football problem is to control what application programs can do. Figure 2 shows how this is accomplished. In this example, we're using QUERY to open a database called orders. We are prompted for a password. How we respond to this prompt will determine what class number we are assigned, and therefore what data we can or cannot access.

Figure 2: Opening a Database with QUERY 
:RUN QUERY.PUB.SYS

HP32216D.03.11  QUERY/3000  TUE, SEP  7, 1999, 11:30 AM 

COPYRIGHT HEWLETT-PACKARD CO. 1976

>DATA-BASE=ORDERS

PASSWORD = >>

MODE = >>1

>FORM SETS

DATA BASE: ORDERS                       TUE, SEP  7, 1999, 11:30 AM

DATA BASE LANGUAGE ATTRIBUTE: NATIVE-3000

                        ITEM   CURRENT   ENTRY    ENTRY   BLOCKING
SETS:             TYPE  COUNT  CAPACITY  COUNT    LENGTH  FACTOR

   CUSTOMER        M     6      201      0        39      10
   DATE-MASTER     A     1      211      0        3       36
   SALES           D     8      1245     0        26      15
In a future article in this series, we'll see that database passwords may not always be handled the way QUERY handles them. In some applications, the passwords may actually be hard-coded into the application program itself. In that case, the level of access that a user has to a database is not a function of what passwords he knows, but of what program(s) he runs. We'll explore password management in more detail in a future article.

In the example shown in Figure 2, we are also prompted for a mode. Like database passwords, the mode may be something you prompt the user for (as QUERY does), but it's more likely to be something that's hard coded in a typical application program. In the previous article of this series, we introduced you to QUERY and brushed over this mode very quickly. IMAGE/SQL allows database access in a variety of different modes, and each one is identified with a number (1, 2, 3, and so forth). The mode that you select in Query determines whether or not you will be able to add records to the database, update existing records, or share the database with other users. It also determines what other users will be able to do. Last time, we used mode 1 without any explanation beyond the fact that mode 1 allowed us to add records to the database (assuming that we've entered an appropriate password).

Now it's time to understand what the mode really means. First of all, the mode is not unique to QUERY. Any program that wants to access an IMAGE database must begin by opening it. In a future article, we'll take a look at the actual MPE intrinsics used to access an IMAGE database from a user-written program, but for now, suffice it to say that everything you learn about Query's Mode prompt will be equally applicable to other application programs. In other words, no matter how you choose to access an IMAGE database, the rules for database access modes still apply.

IMAGE recognizes three different kinds of access to the data it controls:
  1. READ access is just what it sounds like--the ability to read the data stored in the records that you have access to. READ access does not allow you to change any of the data in the database. It is granted with modes 5, 6, 7, or 8.
  2. UPDATE access refers to the ability to update records "in place." In other words, given an existing record, UPDATE access means changing the data in that record (as opposed to creating new records). Having UPDATE access to a database automatically implies that you have READ access as well. Update access is granted with mode 2.
  3. MODIFY access means the ability to add (or delete) entire records (data entries) in the datasets that make up the database. Having MODIFY access implies that you are also allowed to UPDATE and READ the database. Modify access is granted with modes 1, 3 or 4.
Table 1 shows the eight values that can be supplied in response to the Mode prompt in Query. Take a moment now to read through this table and familiarize yourself with the eight values for Mode.

It might seem strange, at first glance, that there are eight possible values for Mode when only three kinds of access are recognized. But that's because each value of Mode determines not only what you are allowed to do, but also what other users of the database are allowed to do at the same time.

Mode 3 and Mode 7 -- Exclusive Access

Probably the simplest modes to understand are modes 3 and 7. Take a look at Table 1. About halfway down the table is a row that explains what mode 3 means. Mode 3 grants you "Modify" access to the database--which means you can add and delete records. The third column of the table is headed "Concurrent Access/Modes Allowed" and it tells what modes other users are allowed to use.

In the case of mode 3, the table tells us that no other concurrent access is allowed. Mode 3 has a "special requirement" (see column 4), which is "Exclusive Access." This means that you cannot open a database in mode 3 if other users currently have the database open. If you try, you'll generate an error. (It won't "queue up" your request until everyone else logs off.)

In order to open a database in mode 3, you must have the database all to yourself. Once you have a database opened in mode 3, all other users are locked out of the database until you're done. Until you close it, any attempt to open the database by anyone else (regardless of what mode they select) will fail.

Mode 7 works exactly the same way as mode 3, with one exception: mode 7 only gives you READ access. Just as with mode 3, mode 7 grants you exclusive access, so you can open in mode 7 only if you have the database all to yourself, and once you have the database open, nobody else can open the database until you close it.

Modes 3 and 7 are rarely used in production application situations. Normally, if one user has exclusive access to a database, it's because you're in a troubleshooting or testing scenario.

Update Access -- Mode 2 and Mode 6

Next, we're going to turn our attention to a slightly more complex scenario. Imagine an application in which you have two classes of users. We'll call them Readers and Updaters.

The first class needs only READ access. For example, think of telephone operators who are taking 411 calls at an information desk. They don't enter any information into the database but are only looking up information in the database and making it available to their callers.

The Updaters class needs UPDATE access. They are allowed to update records in place--but they are not allowed to add or delete any records. So, continuing with our 411 information desk analogy, they can update existing records (possibly changing an address, or a flag that indicates that a particular number is unlisted). But they cannot add a new phone number to the database or delete an old one.

For this type of application, the 2/6 combination of modes is perfect. To understand how it works, imagine for a moment that the first person to open a database is a Reader. This user will use mode 6. Note that mode 6 (see Table 1) not only allows READ access, but it also allows concurrent users to access the database in modes 2, 4, 6, or 8. So if any additional users open the database, they'll need to use one of these modes.

Table 1: Database Access Mode Summary
Access
Mode		 Type of Access
Mode Granted		 Concurrent Access
Modes Allowed		 Special Requirements
1 Modify 1, 5 Modify (with locking) Locking must be used for update or modify.
2 Update 2, 6 Update
3 Modify None Exclusive Access
4 Modify 6 Read
5 Read 1, 5 Modify (with locking) TurboIMAGE/XL does not require locking, but it should be used to coordinate access with users who are modifying the database.
6 Read 2, 4, 6, 8 Modify
7 Read None Exclusive Access
8 Read 6, 8 Read

Now let's suppose that one of our Updaters logs on. The Updaters use mode 2 to open the database, which grants them update access. But it also restricts concurrent access to modes 2 (more Updaters) and 6 (more Readers). In other words, once somebody has the database opened in mode 2, nobody else will be allowed to open the database in any mode other than 2 or 6.

The 2/6 combination of modes is commonly used in applications in which the database is static in the sense that new records are not being added or deleted. If records must be added or deleted, the application program that does it would run after the readers and updaters have logged off. Typically, it would be a batch job that runs at night. But the contents of existing records may be updated during the daytime by any one of a number of users using modes 2 and 6.

Multiple Readers and One Modifier -- Mode 6 and Mode 4

The next scenario we'll consider is similar to the last one. Once again we have multiple people reading the database. But suppose we need to be able to modify the database at the same time. Remember that modifying a database is different from updating it ­ modifying entails adding and deleting records, (as opposed to updating, which only allows you to change existing records). For the moment, let's also allow an important restriction: only one user is going to be doing modifications at a time.

To implement this kind of application, we'll once again use mode 6 for the Readers. But this time we'll use mode 4 for the one and only Modifer. In mode 4, the database is opened for "semi-exclusive modify access." This means that if you open the database in mode 4, then nobody else can open it in mode 4 until you close it. In fact, once you've got the database opened in mode 4, nobody else can open it at all unless they use mode 6 ( which, as we've seen, is read-only).

Our one and only mode 4 user has complete control over database content, just as he did when he was using mode 3 (exclusive access). The difference is that with mode 4, other read-only users are permitted concurrent access to the database.

Multiple Modify Access Users -- Modes 1 and 5

Now let's consider a slightly more complex scenario. Let's assume that we have two classes of users. Once again, we'll have multiple Readers. And once again, we'll assume that there's a need to modify the database simultaneously. But this time out, we'll also have multiple Modifers. Referring to Table 1, you'll find that modes 1 and 5 fit the bill.

But under the column headed "Special Requirements" there are references to something new called "locking." Locking is a subject that we will cover in detail when we begin looking at IMAGE instrinsics in a future article. For now, however, we will conceptually explain what locking is all about.

When multiple users are adding and deleting records in a database at the same time, care must be taken to coordinate these activities. Otherwise, data corruption can result. IMAGE's locking facility is the "traffic cop" that makes it possible for multiple people to access a database simultaneously, while preventing corruption problems.

An application program can lock all or part of a database. IMAGE recognizes three kinds of locks:
  1. Database-level locks
  2. Dataset-level locks
  3. Data entry- (or record-) level locks
If an application program is about to make a change to a database, dataset, or date item, IMAGE expects the application first to lock the resource it wants to change. For example, if the change will affect the structure of the database as a whole, then it expects you to request (and if necessary wait for) a database-level lock. On the other hand, if the change will only affect the structure of one dataset, IMAGE expects you to request (and wait for) a dataset-level lock. For its part, IMAGE won't grant you a lock if some other program or process already has the resource locked.

It may surprise you to learn that IMAGE/SQL does not always require the use of locking. This makes sense in some cases. For example, in modes 3 and 7 (which require exclusive access), locking is not required because the user who has the database opened is the one and only user. He never needs to lock any part of the database, because there's nobody else to lock out.

When the mode you've selected to open the database grants you read-only access, locking would not seem to be necessary, since you aren't changing anything. But referring to Table 1, the "Special Requirements" column for mode 5 tells us:

TurboIMAGE/XL does not require locking, but it should be used to coordinate access with users who are modifying the database.

In other words, even if you are only reading the database, locking may be required if others (operating in mode 1 for example) are modifying the database. The reasons for this will become clearer when we explore locking in more detail in a future article.

We've looked at the various modes in which a database can be accessed, and at some of the combinations in which these modes can be used. Next time, we'll continue our exploration of IMAGE databases by learning exactly how to read data records from an IMAGE database.

George Stachnik works in technical training in HP's Network Server Division.
  ←Part 22  Part 24→
⇑   ⇐   [3khat]3kRanger   [3khat]3kMail   Updated