Java Developers Kit for MPE/iX Version 1.1.7B Release Notes

by Gavin Scott Allegro Consultants, Inc. SIG Java Co-Chairperson for Interex

This release of the Java Developer's Kit (JDK) for MPE/iX includes support for the 1.1.7B version of Sun's JDK, along with significant improvements to the MPE/iX implementation.Highlights of this release relative to version 1.1.5 which shipped with MPE/iX 6.0:

Improved performance
New Just-In-Time Compiler (JIT)
Support for the latest JDK 1.1 version
Reduced resource requirements
Reduced start-up time for the Java VM
Simplified operation




	NOTE: After updating to Release 6.0 Express 1, JDK 1.1.7B must be manually installed by streaming `JINSTJDK.INSTALL.JAVA`

Java, POSIX, and MPE/iX

Java on MPE/iX is based on Sun's reference implementation of Java for UNIX. Because it is a derivative of a UNIX implementation, Java/iX lives almost entirely within the POSIX/HFS environment of MPE/iX. You can invoke Java from the normal MPE Command Interpreter ":" prompt, but you should keep in mind that all filenames operated on by Java will be interpreted as POSIX HFS filenames rather than MPE FILE.GROUP.ACCOUNT names.

The Installation Environment

All of the files that make up the installation of Java/iX reside in the MPE HFS directory starting at:

/usr/local/java/<version>/

For example, after installing 1.1.7B, there will be a directory:

/usr/local/java/jdk1.1.7/

Additionally, each time you install a version of the JDK, a symbolic link named:

/usr/local/java/latest/

will be set to point to that JDK version. Thus the standard way of invoking Java is to go through the /latest/ directory link. In this way, a new version of Java may be installed and applications will immediately start using it. You can keep as many releases of the JDK on the system as you like. Those releases other than the one pointed to by the /latest/ symbolic link will have to be referred to through a path that includes the version number, or in some other fashion. For example, you could create your own symbolic link in the shell as follows:

$ ln -s /usr/local/java/jdk1.1.5 /usr/local/java/production

and then use this /production/ path in your applications. This would allow you to install new releases of the JDK without impacting your current Java applications. Once you have tested a new release, you can simply change the /production/ link to point to the new version.

To remove a version of Java from your system, simply remove all files starting at

/usr/local/java/<version>

For example, if you wish to remove the 1.1.5 version of Java after (or before) installing 1.1.7B, you can use the command:

$ rm -rf /usr/local/java/jdk1.1.5

Important Directories under /usr/local/java/latest

/bin: Contains the user executables. This is the directory that needs to be on your PATH in order to execute Java programs. The actual executables are in /bin/PA-RISC/green_threads/.
/lib: Contains the standard Java classes.zip file and some config files. Shared libraries (XL files) implementing the runtime Java system are in /lib/PA-RISC/green_threads/.

Invoking Java from the POSIX Shell

For convenience, the directory /usr/local/java/latest/bin should be added to your PATH environment variable. This can be done in a user's .profile, or in the system wide /etc/profile.local file with a command such as:

export PATH=/usr/local/java/latest/bin:$PATH

Java can also be invoked by specifying the full path to the executable, as in:

$ /usr/local/java/latest/bin/java HelloWorld

Of course, it is possible to specify a specific version of Java as in:

$ /usr/local/java/jdk1.1.7/bin/java HelloWorld

With version 1.1.7B, the set of environment variables required for Java have been simplified. You no longer need to set SYSNAME or THREADS_FLAG.

CLASSPATH

The CLASSPATH environment variable needs to be set only if you need to specify nonstandard directories other than "." for finding .class files.

If you have not set CLASSPATH, the default path will include these directories:

/usr/local/java/jdk1.1.7/classes
/usr/local/java/jdk1.1.7/lib/classes
/usr/local/java/jdk1.1.7/lib/classes.zip

If you have set CLASSPATH, the following directories will be APPENDED to the list you provide:

/usr/local/java/jdk1.1.7/classes
/usr/local/java/jdk1.1.7/lib/classes
/usr/local/java/jdk1.1.7/lib/classes.zip




	NOTE: If you are running 'appletviewer', and have not set `CLASSPATH`, the current directory (".") will not be included in the default `CLASSPATH` for security reasons.

If all your .class files are in your current working directory, you need not set CLASSPATH at all. If you need to have a directory of your own included in the CLASSPATH, you need only include that directory (and "." if you want it) when you set CLASSPATH. You do not need to include the standard Java classes.zip, or the other "system" directories above, as they will be appended to your CLASSPATH automatically. Note that this is NOT the case if you use the -classpath command line option of the various Java executables. In that case you must specify ALL the locations to be searched. Because of this, the CLASSPATH environment variable method is greatly preferred. JDK 1.2 (the Java 2 platform) will fix this aspect of the -classpath command line option so that it behaves the same as the CLASSPATH environment variable.

LD_LIBRARY_PATH

If you will be invoking native code from your Java program (using JNI, or perhaps the TurboImage class library), Java needs to know where to look for any native libraries that your code tries to load. This is the purpose of the LD_LIBRARY_PATH environment variable. If you are not invoking your own native code, then you do not need to set this variable.

If you do not set LD_LIBRARY_PATH, the default will include the following directory, which contains the standard Java runtime shared libraries:

/usr/local/java/jdk1.1.7/lib/PA-RISC/green_threads

If you set LD_LIBRARY_PATH, then YOUR list of directories will be APPENDED AFTER the above directory.

New Just-In-Time Compiler

The 1.1.7B release of Java/iX includes the latest version of Hewlett-Packard's Just-In-Time (or JIT) Compiler for Java, which provides increased performance for Java programs by transparently converting the interpreted virtual machine bytecodes into native PA-RISC instructions at runtime.

The JIT defaults to being enabled, but can be disabled by passing the -nojit option to the java command, or the -J-nojit option to javac and many of the other JDK commands.

Performance Improvements and Resource Requirement Reductions

With 1.1.7B, several improvements were made to the MPE/iX implementation to reduce the resources required for each instance of the Java Virtual Machine, and to reduce the amount of time and overhead required to start a VM. These changed have resulted in the elimination of approximately half of the start-up overhead.

Here is a summary of differences between 1.1.5 and 1.1.7B:

                           1.1.7                   1.1.5 
-----------------------    --------------------    ------------------------ 
Thread stack allocation    dynamic                 static (39MB stack req.) 
# of Thread stacks         dynamic                 fixed, ~35 
max # of Threads           limited by ;NMSTACK=    fixed, ~35 
Thread stack size          128KB                   1MB 
Min ;NMSTACK for shell     default (2MB) works     NMSTACK=40000000 
Default min heap           256KB                   64MB 
Default max heap           64MB                    64MB 
Out of memory result       Exception thrown        VM aborts with SIGBUS 
# of fork/exec to start    1                       multiple, perhaps many 
Can fork VM?               yes                     no

Thread Stacks

Since the Java Virtual Machine is a multithreaded environment, a stack must be allocated for each thread. Currently Java/iX uses a "Green Threads" package which simulates multiple threads within a single process.

In 1.1.5, storage for thread stacks was statically allocated on the stack at start-up. This resulted in several problems:

The number of threads that was supported was a fixed number.
The VM needed to run with ;NMSTACK=40000000, which required that the user enter the POSIX environment using at least this large of an NMSTACK so that it would be inherited by the eventual Java VM.
The first thread used the highest addresses in the stack, resulting in an apparent 39MB stack size for even the most trivial Java program.
Functions invoked from within the VM that required executing a fork and exec sequence would fail. This included executing shell commands and processes from within Java, and the jdb Java debugger.
Running out of memory could cause an abort rather than throwing an OutOfMemoryException.
The JIT required a large stack size, and could potentially run on any thread, so each thread stack was allocated a full megabyte of storage.

In 1.1.7, the following changes have been made to address these problems:

Thread stacks are now allocated as needed, starting at low addresses in the stack. This has reduced the minimum ;NMSTACK= requirements for the VM from nearly 40MB to around 1.5MB. Because the shell now has a 2MB default NMSTACK, simple Java programs can now be run without any special ;NMSTACK= parameter being specified anywhere.
At start-up, the VM will determine what ; NMSTACK= value is in effect, and will allow allocation of as many thread stacks as will fit within this limit. When the limit is exceeded, an OutOfMemoryException will be thrown. The default ;NMSTACK= value of 2MB for the POSIX shell means that approximately three threads can be created before a larger ;NMSTACK= will need to be specified by the user. This is enough to run the javac java compiler and non-threaded java programs. Programs that use AWT create several internal threads and will require a larger VM STACK. The JAVA executable has a default NMSTACK of 10MB, which will be in effect if the VM is invoked directly from the CI, and should be enough for most applications.
Ordinary thread stacks have been reduced from 1MB to 128KB in size. The JIT now has its own dedicated stack which is sized as needed by the particular JIT version.
Functions like Runtime.exec are now able to successfully fork from inside the VM.

Setting the NMSTACK Size for the Java VM Process

As noted above, it is no longer necessary to run the shell with a 40 million byte stack limit in order to then run Java. Programs which need to create a large number of threads will need a larger stack limit.

When a process is created in the "MPE style" by calling the CREATE or CREATEPROCESS intrinsics, the new process will have its ;NMSTACK= limit set by the default value linked into the program file, unless it is overridden by a CREATEPROCESS option. So if you are invoking the Java VM directly from the CI through a command file like JAVA.PUB.SYS, or the :RUN command, the new VM process will get the 10MB NMSTACK value of the JAVA executable, which should be enough for approximately 65 threads.

When a new process is started via the POSIX fork and exec sequence, the new son process inherits the NMSTACK limit of its father process. The NMSTACK limit of the last process that was created by CREATE[PROCESS] affects all forked descendants of that process. To see why, we have to look at what fork and exec do. fork creates a new process that is an exact copy of the calling process, so obviously it should and must have the same NMSTACK limit as the old process that it is a copy of. Once the fork is complete, exec is called to change the program being executed by the process from the copy of the original to the new program we wish to run. Unfortunately at this point the stack for the process has been set up (based on the copy of the original process) and there is no opportunity to change it, even if the new program would like to specify a different limit.

When executing program from inside the POSIX shell, the fork and exec sequence is used by the shell (and most other "POSIX" type programs), which means that the same NMSTACK limit will apply to every one of these processes. This means that whatever ;NMSTACK= was in effect when you entered the shell from the CI using the :RUN SH.HPBIN.SYS command (or equivalent) will apply to everything (like Java) that you run from inside the shell.

If you will be invoking Java from within the shell, and you need more than approximately three threads for your program. Then you will need to arrange to have specified a larger ;NMSTACK= value when you ran the first shell. Java requires approximately 1.5MB of NMSTACK for a single thread program, and 128KB more for each additional thread.

If you want to use 100 simultaneous threads in your Java program, you would want to enter the shell with a command such as:

:RUN SH.HPBIN.SYS;INFO="-L";NMSTACK=14000000

Heap Changes

The default minimum heap (java -ms option) has been changed from 64MB to 256KB in version 1.1.7B. The default maximum value is still set to 64MB, which is under the 80,000,000 byte system default ;NMHEAP= limit and thus does not involve the problems of ;NMSTACK=. Should you need more than ~70MB of Java heap storage, you will need to both specify the larger limit using the java -mx option AND specify a larger ;NMHEAP= value in the same way that a larger ;NMSTACK= is specified.

The smaller default minimum heap value reduces the memory management overhead and start-up time for the VM.

Extra Process Elimination at Start-up Time

One of the biggest reductions in start-up overhead was realized by eliminating the extra fork/exec sequences of the standard VM start-up shell scripts.

On UNIX systems, processes are very cheap to start, and every time the shell wants to run a program it simply forks and execs to do it. On MPE however, this is a VERY expensive process.

In the JDK 1.1.5 version of Java/iX, if you want to execute the "javac" command to compile a program, here's what happens when the user types:

$ javac -g HelloWorld.java

The shell reads /usr/local/java/jdk1.1.5/bin/javac which is a symbolic link to /usr/local/java/jdk1.1.5/bin/.java_wrapper.
The shell forks a copy of it self to read and execute the . java_wrapper script.
The .java_wrapper script executes three POSIX commands to break apart the $0 argument to the script. Each of these requires a fork and exec to create a new process to run the command!
The .java_wrapper forks a new shell and it execs /usr/local/java/jdk1.1.5/bin/PA-RISC/green_threads/javac which is yet another shell script.
The /usr/local/java/jdk1.1.5/bin/PA-RISC/green_threads/javac script invokes another script (another fork/exec) to break up the parameters to the javac command. For EACH parameter, this script does MORE fork/execs to help chop up the input.
Finally the /usr/local/java/jdk1.1.5/bin/PA-RISC/green_threads/javac script execs /usr/local/java/jdk1.1.5/bin/PA-RISC/green_threads/java, which is the actual VM executable, passing it the name of the javac class to be executed to run the compilation.

The result of all this is that the user spends several seconds waiting while the system forking repeatedly.In 1.1.7B, all of the above nonsense has been replaced by a single program. All of the extra fork/exec sequences are gone. The new start-up sequence looks like: