HP 3000 Manuals

Overview of the Export Action [ HP Performance Collection Software User's Manual (for MPE Systems) ] MPE/iX 5.0 Documentation

HP Performance Collection Software User's Manual (for MPE Systems)

Overview of the Export Action

EXPORT copies Performance Collection Software log file data into a form
that can be easily accessed without passing through the HP LaserRX/MPE
analysis programs. Any valid raw or extracted log file can be the source
of this data.

The process is summarized in figure 5-1.

Figure 5-1. The Export Function

Exported files can be used in a variety of ways, such as custom graphics
packages, databases, and user-written analysis programs. The main
advantage of using exported files rather than the analysis programs to
export the log file data lies in the significant time saved by not having
to transport the data to the PC.

How to Export Data

In the simplest form, you can export data by specifying the default log
file and the default report file, then starting the export. The default
report file allows you to export files similar to the current HP
LaserRX/MPE EXPORT LOGFILE function.

:RUN EXTRACT
LOGFILE LOGGLOB.SCOPE.SYS
REPORT REPTFILE.SCOPE.SYS
EXPORT

Exported data is in a file called XFERGLOB in a format suitable for
loading into a spreadsheet.

If you want to export something other than this default set of data, you
can use various commands and files in conjunction with the Export
command.

* You can export the following types of data:

Global (Five-minute and hourly summaries.)

Application (Five-minute and hourly summaries.)

Process (One-minute details.)

Disc (Daily data points.)

Configuration (One record containing Performance
Collection Software PARM file information,
etc., for each time the collector started.)

* You can specify what data items are needed for each type of data.
Refer to the tables under "Data Items for Exporting Data" later in
this section for a list of the data items for each type of data.

* You can choose to specify starting and ending dates for the data
along with the SHIFT and WEEKEND EXCLUSION filters.

* You can specify the format of the exported data in an ASCII report
file. This file can be created using any standard host editor
program or you can use the default file, REPTFILE.

Sample Export Tasks

Several sample report files are furnished with the Performance Collection
Software. They may be used initially to perform common reporting tasks
or as a starting point for custom tasks.

* Generate a CPU and DISC report on a printer.
The REPTHIST report file contains the specifications to generate a
character graph of CPU and DISC usage for a system over time.
This graph is composed of printable characters and may be printed
on any device capable of 132 column printing. This example
generates a graph of the last seven days on a system and should
take approximately two pages (34 pages if five-minute detail is
selected instead of hourly summaries).

:HELLO SCOPE.SYS,SCOPE
:RUN EXTRACT
REPORT REPTHIST
GLOBAL SUMMARY
START TODAY-7
EXPORT
EXIT
:COMMENT At this point the data is in a file XFERGSUM
:COMMENT Now copy it to the printer
:FILE PRINTER;DEV=LP
:FCOPY FROM=XFERGSUM;TO=*PRINTER
:PURGE XFERGSUM

* Summarize the process data and list the top CPU consumers for an
interval. The PROCJOB job stream has been furnished to use the
EXTRACT program and other standard MPE programs such as EDITOR,
FCOPY and SORT to perform this task. It will print a report
showing all processes which ran to completion yesterday, sorted by
the amount of CPU each one used. The report file used by PROCJOB
is called REPTPROC.

To print the top processes on your system for yesterday:

:STREAM PROCJOB.SCOPE.SYS (You may need to add passwords)
Examine the job's $STDLIST for the report

* Summarize the process data and list the top programs, as ranked by
amount of CPU used. While similar to the previous task, this one
adds the need to combine multiple executions of a program file,
multiple processes, into a summary for the program file. A simple
program has been written to combine multiple executions of a
program into a single report line. The name of this program is
TOPCPU and its source, in COBOL, is in the file TOPCPUS. The
TOPJOB job stream will produce a report of the top programs which
ran on your system yesterday. It uses the REPTTOP report file.

To print the top programs on your system for yesterday:

:STREAM TOPJOB.SCOPE.SYS (you may need to add passwords)
Examine the job's $STDLIST for the report

* Produce a customized export file. If one of the previous tasks is
similar to what you desire, you can make a copy of the report
file, then customize it. If you want to create a totally new
export format then you can copy the report file REPTALL.SCOPE.SYS
and modify it. The REPTALL file contains every possible item for
each different data type so all you need to do is delete those
items which are not of interest to you. This is easier than
typing a report file from scratch.

_________________________________________________________________

NOTE Using the REPTALL report as furnished will probably cause
warnings saying not all the data would fit into the maximum
width data record. This is normal and simply indicates that
you should select a subset of data to export.

_________________________________________________________________

Export Data Files

* Export can create up to seven exported files, depending on the
types of data and the report file selected. A log file will not
be created if items were not selected for the type of data in the
report file or if the EXTRACT program did not receive the proper
command prior to the EXPORT command.

All seven data files will be purged at the beginning of an EXPORT
function regardless of whether or not they will be recreated
later. Be careful not to have other files with the following
names in your logon group when performing the EXPORT function.

The EXPORT log file names are:

XFERGLOB Global Detail Data file.

XFERGSUM Global Hourly Summary Data file.

XFERAPPL Application Detail Data file.

XFERASUM Application Hourly Summary Data
file.

XFERPROC Process Detail Data file.

XFERDISC Disc Space Data file.

XFERCONF Configuration Data file.

You may use file equations to redirect any of these files to
another disc file name. Redirection to nondisc files is not
supported currently.

In summary, the following commands affect the actions of the Export
function:

GLOBAL START LOGFILE
APPLICATION STOP REPORT
PROCESS SHIFT
DISCSPACE
CONFIGURATION

For details on these commands refer to the command reference earlier in
this chapter.

Creating a Report File

The report file contains the following information:

[ASCII ]
REPORT [TITLE] title string FORMAT [DATAFILE] HEADINGS [ON ]
[BINARY ] [OFF]

SEPARATOR=char datatype items

where:

REPORT TITLE (Prints an optional character string and
headings.) The following page has more details.

FORMAT (Selects the data formats.)

ASCII An ASCII format file is similar to the HP
LaserRX/MPE analysis software export to DATAFILE
format. Files in ASCII format are the best
files for copying to a printer or terminal.

DATAFILE An ASCII-format file in which all non-numerical
fields are enclosed in double quotes. Since
double quotation marks makes it impossible
maintain column alignment, files in DATAFILE
format are not suitable for printing directly.
DATAFILE format is the easiest format to import
into most PC spreadsheets and graphics packages.

BINARY Binary format is a more compact format that
represents numerical values as binary integers.
It is the most suitable format for input into
user-written analysis programs. Since it
requires the least amount of conversion, it
maintains the highest metric accuracy, but it is
not suitable for printing directly.

Table 5-1. (cont.)

HEADINGS (Selects whether or not to include column
headings in the data file (HEADINGS=ON or
HEADINGS=OFF).)

SEPARATOR (Selects the character that is printed between
each field in the DATAFILE format.) The default
separator is a blank space, but many programs
prefer the field separator to be a comma. You
may set the separator to any printing or
nonprinting character.

DATA TYPE (Selects one of the exportable data types:
GLOBAL, APPLICATION, PROCESS, DISCSPACE, or
CONFIGURATION.) This starts a section of the
report file that lists the data items to be
copied when this type of data is exported.

ITEMS (Specifies the data to be included in the
exported file.) Item names are listed, one per
line, in the order you want them listed in the
resulting file. You must select the proper DATA
TYPE before listing any ITEMS.

You may include item lists for as many data
types as you wish in the same report file. Each
data type will be referenced only if you choose
to export that type of data. See the tables
later in this chapter for the data items for
each data type.

You may have more than one report file on your system. Each one can
define a set of exported file formats to suit a particular user's needs.
You specify the report file to be used with the REPORT command when you
run the EXTRACT program.

The following items may be substituted in the REPORT TITLE string:

!DATE (Date the EXPORT function was performed.)

!TIME (Time the EXPORT function was performed.)

!LOGFILE (The fully qualified name of the source
performance log file.)

!COLLECTOR (Name and version of the Performance Collector
program.)

!SYSTEM_ID (Identifier of the system that collected the
data.)

For example, the string

REPORT "Export !SYSTEM_ID data from !LOGFILE on !DATE !TIME"

would generate a report title similar to

Export COOKIE data from LOGGLOB.SCOPE.SYS on 02/02/91 08:30 AM

An Example of Exporting Data

Example: You want to export GLOBAL and APPLICATION data at a rate of one
data point per hour for use in creating a custom graph or report. Take
the following steps:

1. First, you must determine what data items you will need from each
data type and in what format should you access them. For this
example, assume we will be graphing Global Queue Depths and
Application Response Times, you would like an ASCII file without
headings, and each field will be separated by commas.

2. Create and save the following ASCII file. Call it REPORT1.

REPORT "Sample Report File (REPORT1)"
FORMAT ASCII
HEADINGS OFF
SEPARATOR=","

DATA TYPE GLOBAL

CPUQUEUE
DISCQUEUE
MEMORYQUEUE
IMPEDEQUEUE

DATA TYPE APPLICATION

APPLICATION
TRANSACTIONS
FIRSTRESP
PROMPT

3. Run the EXTRACT program.

:RUN EXTRACT
Enter command (or press softkey)

4. Select the report file generated.

REPORT REPORT1

5. Select GLOBAL SUMMARY data and APPLICATION SUMMARY data using
standard EXTRACT program commands.

GLOBAL SUMMARY
APPLICATION SUMMARY

6. Now enter EXPORT--it means "GO".

EXPORT
ENTER THE LOG FILE NAME (LOGGLOB.SCOPE.SYS)

7. Since you didn't tell the program from where it should get the
performance data, the program will prompt you. In this example
the default log file is correct, just press RETURN.

EXPORTING GLOBAL DATA .........50%......100%
EXPORTING APPLICATION DATA ....50%......100%

The exported file contains 31 days of data from 09/01/91 to 10/01/91

Examined Exported
Data Type Records Records Space
----------------------- --------- --------- ---------
GLOBAL 8817 0 0.00 MB
GLOBAL SUMMARIES 736 0.20 MB
APPLICATION 28491 0 0.00 MB
APPLICATION SUMMARIES 2560 0.71 MB
---------
0.91 MB

You are finished. You have just created two files--XFERGSUM and
XFERASUM--that contain the global and application summary data in the
format you specified.

Data Items for Exporting Data

The following data items are available for Data Type GLOBAL:

Table 5-2. EXPORT Items for Data Type GLOBAL

Record Identification Metrics

RECORD_TYPE ASCII field to identify this record type
"GLOB".

DATE Date in MM/DD/YY format (or custom NLS
date).

TIME Time in HH:MM 24-hour format.

DAY Julian day-of-the-year (1-366).

YEAR Year (such as 1991).

DATE_SECONDS Date in UN*:X format (in seconds, since
January 1, 1970).

INTERVAL Time included in this sample (in seconds).

SAMPLES Number of individual data samples averaged
in this data.

NUMBER_OF_DISCS Number of disc drives configured on the
system.

BLANK An empty field used as a spreadsheet place
holder.

Summary Metrics

CPU_TOTAL Average overall CPU usage during the
interval (percentage of total).

CPU_SECONDS Average overall CPU usage during the
interval (time, in seconds).

DISC_TOTAL Average overall physical disc IO rate
(IOs/second).

DISC_IO Average overall physical disc IO count.

CPU_HISTOGRAM A 60-character-wide histogram of CPU
components.

DISC_HISTOGRAM A 60-character-wide histogram of DISC IO
components.

Table 5-2. EXPORT Items for Data Type GLOBAL (cont.)

CPU Metrics

CPU_SESSION CPU usage by interactive sessions
(percentage of total).

CPU_SESSION_SECONDS CPU usage by interactive sessions (time, in
seconds).

CPU_JOB CPU usage by batch jobs (percentage of
total).

CPU_JOB_SECONDS CPU usage by batch jobs (time, in seconds).

CPU_SYSTEM CPU usage by system processes (percentage of
total).

CPU_SYSTEM_SECONDS CPU usage by system processes (time, in
seconds).

CPU_PAUSED Time CPU was idle and Disc IO was occurring
(percentage of total).

CPU_PAUSED_SECONDS Time CPU was idle and Disc IO was occurring
(in seconds).

CPU_CACHE CPU usage by Disc Caching (percentage of
total; MPE V only).

CPU_CACHE_SECONDS CPU usage by Disc Caching (time, in seconds;
MPE V only).

CPU_DISPATCH CPU usage by the Dispatcher (percentage of
total; MPE/iX only).

CPU_DISPATCH_SECONDS CPU usage by the Dispatcher (time, in
seconds; MPE/iX only).

CPU_ICS CPU usage for other activities such as
interrupts (percentage of total).

CPU_ICS_SECONDS CPU usage for other activities such as
interrupts (time, in seconds).

CPU_MEMMGR CPU usage for memory management (percentage
of total).

CPU_MEMMGR_SECONDS CPU usage for memory management (time, in
seconds).

CPU_IDLE CPU usage when processor was not busy or
paused (percentage of total).

CPU_IDLE_SECONDS CPU usage when processor was not busy or
paused (time, in seconds).

PROCESSOR1_BUSY Percentage busy on first system processor.

PROCESSOR1_SECONDS Time busy on first system processor (in
seconds).

PROCESSOR2_BUSY Percentage busy on second system processor
(MPE/iX MP only).

PROCESSOR2_SECONDS Time busy on second system processor (in
seconds, MPE/iX MP only).

PROCESSOR3_BUSY Percentage busy on third system processor
(MPE/iX MP only).

PROCESSOR3_SECONDS Time busy on third system processor (in
seconds, MPE/iX MP only).

PROCESSOR4_BUSY Percentage busy on fourth system processor
(MPE/iX MP only).

PROCESSOR4_SECONDS Time busy on fourth system processor (in
seconds, MPE/iX MP only).

Table 5-2. EXPORT Items for Data Type GLOBAL (cont.)

Disc Metrics

DISC_LOGICAL Logical disc IO rate (IOs/second).

DISC_LOGICAL_IO Number of logical disc IO transfers
(kilobytes transferred).

DISC_SESSION Physical disc IO rate by interactive
sessions (IOs/second).

DISC_SESSION_IO Number of physical disc transfers by
interactive sessions (kilobytes
transferred).

DISC_JOB Physical disc IO rate by batch jobs
(IOs/second).

DISC_JOB_IO Number of physical disc IO transfers by
batch jobs (kilobytes transferred).

DISC_SYSTEM Physical disc IO rate by system processes
(IOs/second).

DISC_SYSTEM_IO Number of physical disc transfers by system
processes (kilobytes transferred).

DISC_MEMMGR Physical disc IO rate for memory management
(IOs/second).

DISC_MEMMGR_IO Number of physical disc transfers for memory
management (kilobytes transferred).

DISC_LOGLREAD Logical disc read rate (IOs/second).

DISC_LOGLREAD_IO Number of logical disc read transfers
(kilobytes transferred).

DISC_LOGLWRITE Logical disc write rate (IOs/second).

DISC_LOGLWRITE_IO Number of logical disc write transfers
(kilobytes transferred).

DISC_PHYSREAD Physical disc read rate for user files
(reads/second).

DISC_PHYSREAD_IO Number of physical disc reads for user
files.

DISC_PHYSWRITE Physical disc write rate for user files
(writes/second).

DISC_PHYSWRITE_IO Number of physical disc writes for user
files.

DISC_MEMREAD Physical disc read rate for memory
management (reads/second).

DISC_MEMREAD_IO Number of physical disc reads for memory
management.

DISC_MEMWRITE Physical disc write rate for memory
management (writes/second).

DISC_MEMWRITE_IO Number of physical disc writes for memory
management.

DISC_UTILIZATION Peak disc utilization (busiest disc drive;
percentage of total).

DISC_UTIL_SECONDS Peak disc utilization (time busy on busiest
drive, in seconds).

DISC_KBYTE/SECOND Physical disc transfer rate (kilobytes per
second).

DISC_KBYTE_COUNT Number of physical disc transfers (kilobytes
transferred).

Table 5-2. EXPORT Items for Data Type GLOBAL (cont.)

Process Queue Depths (Load Factors)

CPUQUEUE Average number of processes waiting for or
using CPU.

DISCQUEUE Average number of processes waiting for DISC
transfers.

MEMORYQUEUE Average number of processes waiting for
MEMORY.

IMPEDEQUEUE Average number of processes waiting for
IMPEDES (locks).

Job/Session Count Metrics

NUM_JOBS Average number of logged-on batch jobs.

NUM_SESSIONS Average number of logged-on interactive
sessions.

ACTIVE_JOBS Average number of batch jobs using CPU
resources.

ACTIVE_SESSIONS Average number of interactive sessions using
CPU resources.

JOBS_COMPLETED Number of batch jobs that completed during
the interval.

JOBS_RUNTIME Average run time for completed batch jobs
(in seconds).

JOB_PROC_COMPLETED Number of batch processes that completed
during the interval.

JOB_PROC_RUNTIME Average run time for completed batch
processes (in seconds).

SESSIONS_COMPLETED Number of sessions that completed during the
interval.

SESSIONS_RUNTIME Average run time for completed sessions (in
seconds).

SESS_PROC_COMPLETED Number of session processes that completed
during the interval.

SESS_PROC_RUNTIME Average run time for completed session
processes (in seconds).

Terminal Transaction Metrics

TRANSACTIONS Number of completed terminal transactions
during the interval.

TRANSACT/MIN Terminal transaction rate (transactions per
minute).

TRANSACT/HOUR Terminal transaction rate (transactions per
hour).

THINKTIME Average think time for terminal transactions
(in seconds).

FIRSTRESP Average first-response time for transactions
(in seconds).

PROMPT Average response-to-prompt time for
transactions (in seconds).

Table 5-2. EXPORT Items for Data Type GLOBAL (cont.)

Terminal Transaction Distributions. Each metric is an array of 10 items.

DISTRIBUTION_FIRST First-response time distribution (10
values).

DISTRIBUTION_PROMPT Response-to-prompt distribution (10 values).

DISTRIBUTION_THINK Transaction think-time distribution (10
values).

Individual Disc Metrics. The following fields will be repeated once for every disc on the
system.

@DISC_LOGLREAD Logical read rate (IOs/second).

@DISC_LOGLREAD_IO Number of logical reads.

@DISC_LOGLWRITE Logical write rate (IOs/second).

@DISC_LOGLWRITE_IO Number of logical writes.

@DISC_PHYSREAD Physical read rate (IOs/second).

@DISC_PHYSREAD_IO Number of physical reads.

@DISC_PHYSWRITE Physical write rate (IOs/second).

@DISC_PHYSWRITE_IO Number of physical writes.

@DISC_MEMREAD Memory management read rate (IOs/second).

@DISC_MEMREAD_IO Number of memory management reads.

@DISC_MEMWRITE Memory management write rate (IOs/second).

@DISC_MEMWRITE_IO Number of memory management writes.

@DISC_UTILIZATION Percentage disc utilization.

@DISC_KBYTE/SEC Physical disc transfer rate (kilobytes per
second).

@DISC_KBYTE_COUNT Number of physical disc transfers (kilobytes
transferred).

APPLICATION Data Type.

Application data type can generate two different record layouts. The
first layout writes one record for every application that was active
during the time interval. Each application is in a separate record.

The second layout writes only one record for each interval. In this
layout, a section of the record is reserved for every application. If
the application was not active during the interval then its fields will
contain zeros.

You determine which application layout you will use by the metrics you
choose. The application metrics will be listed in three sections. The
first section ("Record Identification Metrics") can be used in either
layout. Other application metrics may be selected from only one of the
next two sections ("Single Application" and "Multiple Application"
layouts, respectively).

Used with either application layout:

Table 5-3. EXPORT Items for Data Type APPLICATION (Common)

Record Identification Metrics

RECORD_TYPE ASCII field to identify this record type
"APPL".

DATE Date in MM/DD/YY format (or Custom NLS
date).

TIME Time in HH:MM 24-hour format.

DAY Julian day-of-the year (1-366).

YEAR Year (such as 1991).

DATE_SECONDS Date in UN*X format (in seconds, since
January 1, 1970).

NUMBER_OF_APPLS Number of applications defined.

NUMBER_OF_DISCS Number of disc drives configured on the
system.

Table 5-4. EXPORT Items for Data Type APPLICATION (Single Layout)

Used with single application layouts:

Application ID Metrics

INTERVAL Time covered by this record (in seconds).

SAMPLES Number of samples averaged into this record.

APPLICATION_NO Sequential application number (matches PARM
file).

APPLICATION Application name (20-byte ASCII).

BLANK An empty field used as a spreadsheet place
holder.

Summary Metrics

CPU_TOTAL CPU usage by the application during the
interval (percentage of total).

CPU_SECONDS CPU time used by the application during the
interval (in seconds).

DISC_TOTAL Physical disc IO rate (IOs/second).

DISC_IO Number of physical disc IOs.

CPU Metrics

CPU_LINEAR CPU usage while in the linear execution
queue (percentage of total).

CPU_LINEAR_SECONDS CPU usage while in the linear execution
queue (time, in seconds).

CPU_CS CPU usage while in the "CS" execution queue
(percentage of total).

CPU_CS_SECONDS CPU usage while in the "CS" execution queue
(time, in seconds).

CPU_DS CPU usage while in the "DS" execution queue
(percentage of total).

CPU_DS_SECONDS CPU usage while in the "DS" execution queue
(time, in seconds).

CPU_ES CPU usage while in the "ES" execution queue
(percentage of total).

CPU_ES_SECONDS CPU usage while in the "ES" execution queue
(time, in seconds).

Table 5-4. EXPORT Items for Data Type APPLICATION (Single Layout) (cont.)

Disc Metrics

DISC_LINEAR Physical disc IO rate while in linear
execution queue (IOs/second).

DISC_LINEAR_IO Number of physical disc IOs while in the
linear execution queue.

DISC_CS Physical disc IO rate while in the "CS"
execution queue (Ios/second).

DISC_CS_IO Number of physical disc IOs while in the
"CS" execution queue.

DISC_DS Physical Disc IO rate while in the "DS"
execution queue (IOs/second).

DISC_DS_IO Number of physical disc IOs while in the
"DS" execution queue.

DISC_ES Physical disc IO rate while in the "ES"
execution queue (IOs/second).

DISC_ES_IO Number of physical disc IOs while in the
"ES" execution queue.

Process Queue Depths

CPUQUEUE Average number of processes waiting for or
using CPU.

DISCQUEUE Average number of processes waiting for
DISC.

MEMORYQUEUE Average number of processes waiting for
MEMORY.

IMPEDEQUEUE Average number of processes waiting for
IMPEDES (locks).

Process Count Metrics

AVE_PROCESSES Average number of processes in the
application.

ACTIVE_PROCESSES Average number of processes that used CPU.

PROCESSES_COMPLETED Number of application processes that
completed.

PROCESSES_RUNTIME Average run time of completed processes (in
seconds).

Terminal Transaction Metrics

TRANSACTIONS Number/Rate of terminal transactions
completed during the interval.

TRANSACTION_COUNT Number of terminal transactions completed
during the interval.

THINKTIME Average transaction think time (in seconds).

FIRSTRESP Average transaction time-to-first-response
(in seconds).

PROMPT Average transaction response-to-prompt time
(in seconds).

SERVICE_LEVEL Percentage of transactions that met service
level.

SERVICE_LEVEL_COUNT Number of transactions that met service
level.

Table 5-4. EXPORT Items for Data Type APPLICATION (Single Layout) (cont.)

Average process Wait State Metrics

STOPCPU Time processes spent waiting for CPU
(percentage of total).

STOPDISC Time processes spent waiting for DISC
(percentage of total).

STOPSWAP Time processes spent waiting for MEMORY
(percentage of total).

STOPIMPEDE Time processes spent IMPEDED (locked)
(percentage of total).

STOPOTHERIO Time processes spent waiting for other IO
(percentage of total).

STOPTERM Time processes spent waiting for terminal
input (percentage of total).

Individual Disc Metrics. The following fields will be repeated once for each disc on the
system:

@DISC_LOGLREAD Logical read rate (IOs/second).

@DISC_LOGLREAD_IO Number of logical reads.

@DISC_LOGLWRITE Logical write rate (IOs/second).

@DISC_LOGLWRITE_IO Number of logical writes.

@DISC_PHYSREAD Physical read rate (IOs/second).

@DISC_PHYSREAD_IO Number of physical reads.

@DISC_PHYSWRITE Physical write rate (IOs/second).

@DISC_PHYSWRITE_IO Number of physical writes.

MPE/iX 5.0 Documentation