DESCRIPTION
A processor receives an interrupt when either the
processor's interrupt pin is asserted (for line based interrupts)
or if a processor detects an interrupt message bus transaction on
the system bus (for transaction based interrupts).
Interrupts from the interface cards can be line or transaction
based. Interrupts are routed to different processors during boot
time.
The
intctl
command is a tool that allows a performance expert to
display and modify these interrupt assignments.
The tool only supports migration of external device interrupts.
The performance analyst
can also save and restore the interrupt configuration.
If interrupt migration process completes
successfully, a message is logged to the console and/or to
the
/var/adm/syslog/syslog.log
file.
intctl
resides in
/usr/contrib/bin,
and the command can be executed only by
the superuser.
The
intctl
command is not a general system administration command.
It should be used only by performance tuning experts with a high
level of system knowledge. The performance specialist can use the
intctl
command to view the interrupt configuration of the system
and modify the interrupt assignments of the CPUs to re-distribute
the system load across the CPUs.
intctl
is synchronized with other High Availability (HA) events
happening simultaneously on the system. An HA event can be a PCI
OLA/R or Processor allocation/de-allocation. If any of these
events are happening when
intctl
is trying to display interrupt
information or is trying to migrate an interrupt to a CPU,
intctl
will exit with the error message below,
and the user should retry the
intctl
command:
"Another HA event is in progress, try again!"
Interrupt migration is not enabled on workstations.
Also non-MP safe drivers do not support interrupt migration.
The tool will display an error message if the user
tries to move the interrupts of a non-MP safe driver to a
different CPU.
On a system with virtual partitions (vPars),
intctl
will only
display CPUs in the current partition.
Using the
-b
option, the command can be used to balance interrupt distribution
on a system. Interrupt assignments in a given system may not be distributed
in a balanced manner.
Most of the time, imbalance in distribution is caused by CPU migrations.
These migrations may cause the interrupts to get assigned to CPUs
available in the system in a non-optimal fashion and they will not
get distributed when more CPUs become available.
By default, HP-UX will distribute interrupts at boot time based on
the round robin method.
Each interrupt owned gets assigned to the
available CPUs in a round robin fashion.
However, CPU migrations, which
may happen because of Work Load Manager (WLM) configuration,
vPar administration activity, and Instant Capacity (iCAP/TiCAP)
administration activity, can cause the interrupts to be assigned to a
smaller set of CPUs causing an imbalance and thus a non-optimally
configured system.
Using the
-b
option allows the user to manually balance the interrupt distribution
of the system.
Users can choose one of these two balancing algorithm of their
choice to balance
interrupts:
driver_weight
The default balancing algorithm used by
intctl
is
driver_weight,
which associates weights to each driver based on its interrupt frequency
and balances the system such that each CPU is loaded with a similar
average weight from the interrupt load perspective.
round_robin
Another supported algorithm is
round_robin,
which assigns interrupts to the available CPUs in a round robin fashion.
This is similar to the HP-UX default boot time interrupt distribution method, but
the interrupt assignments may differ because of the difference in
the way I/O cards and CPUs are discovered.
driver_weight
is a better choice of algorithm for systems having I/O cards that
demand largely varying range of interrupt processing needs.
Hence
driver_weight
is chosen as the default algorithm.
In systems where all
I/O cards demand similar interrupt processing capacity or when
it becomes difficult to determine interrupt load generated by
each driver, then the
round_robin
algorithm can be used.
Administrators can also configure automatic balancing of interrupts at
periodic intervals.
Balancing will be performed only if there is an
interrupt distribution imbalance.
This is desirable in a dynamic CPU
migration environment such as WLM (Work load Manager).
Refer to
intrbald(1M)
for more details.
Several settings are provided for managing balancing of interrupts.
This information can be provided using the command line
options or can be persistently configured in the configuration file
/etc/intctl.conf,
see the
Interrupt Configuration File
section below.
Options
By default, the command displays interrupt information about all
the interface cards on the system.
intctl
recognizes the following options:
- -a algorithm
Balancing of interrupts may be performed any time during system up
time to reduce CPU overload because of interrupt handling.
Users have several options to control balancing of interrupts.
There are two algorithms supported to balance interrupt
distribution.
Driver weight
(-a driver_weight)
based approach.
This is the default algorithm used to balance interrupts when no
algorithm is specified.
The default can be set by changing
INTCTL_DEFAULT_ALGO
in the configuration file
/etc/intctl.conf.
Each driver is given a weight based on the number of interrupts
it may generate.
Balancing operations ensure that each CPU is
loaded (from interrupt load perspective) with a comparable total
driver weight.
These weights can be between
0
and
INT_MAX
(see
limits(5)).
Most of the HP-UX drivers are already defined in the configuration
file section
INTCTL_DRIVER_WEIGHTS.
Users can modify or override these driver weights, but they should
make sure not to set unrealistic driver weights without knowing the
amount of interrupt load the driver could generate.
Round robin
(-a round_robin)
approach.
Each interrupt in the system will be assigned an available CPU in
round robin fashion.
This balancing approach can be used when it becomes difficult to
differentiate drivers based on their interrupt load.
Compared to the driver weight based approach, round robin could
result in more interrupt migrations while balancing interrupts.
- -b
Balance the interrupt distribution of the system by
performing the least number of migrations that will distribute
interrupt load across a specified set of CPUs.
- -c cpu_id
By itself (without any other options),
display interrupt information about
the specified CPU.
When used with the
-M
option,
-c cpu_id
specifies the CPU ID of
the CPU to which the interrupt is to be moved.
- -C class
Display interrupt information about all the
interface cards belonging to the specified class.
Can be used with
the
-H hw_path
option
to display interrupt information about the interface card
under the
hw_path
that belongs to the specified
class.
- -F
Produce a compact listing of fields separated
by colons.
- -h
Display the usage of the command.
- -H hw_path
Display interrupt information about all
interface cards connected at the specified
hardware path.
For hardware paths
/
and
0,
intctl
prints the interrupt information about
all the interface cards on the system.
When used with the
-C class
option,
-H
displays information
about all interface cards connected to the path
and which belong to the specified class.
When used with the
-M
option,
-H hw_path
specifies the hardware
path of the interrupt that needs to be moved
to a different CPU.
- -i
While balancing interrupts, it may be desirable to avoid
some interrupt assignments based on the I/O cards, CPUs, and so on.
Multiple parameters can be specified by
using the
-i
option multiple times.
NOTE:
Specifying many parameters may reduce the scope of balancing
interrupts and may cause interrupt distribution imbalance.
Ignore I/O Card Interrupt
(-i io:hw_path:intr_id)
While balancing interrupts, interrupt
intr_id
associated with this
I/O card at the hardware path
hw_path
will not be migrated.
intr_id
represents a specific interrupt id or all interrupt ids of the
specified I/O card if
-1
is used.
These values can also be specified in the configuration file section
INTCTL_HW_IGNORE.
Ignore CPU
(-i cpu:hw_path)
None of the interrupts assigned to the CPU with the specified
hw_path
will be migrated while balancing interrupts.
Also, no new interrupts will be assigned to
these CPUs while balancing interrupts.
The same information can be specified in the
INTCTL_CPU_IGNORE
section of the configuration file.
Ignore Driver
(-i drv:driver_name)
None of the I/O cards claimed by the driver
driver_name
will be selected for interrupt migrations while balancing interrupts.
Care needs to be taken while specifying a
driver_name
as one driver
could possibly claim multiple I/O cards and specifying such a
driver will reduce the scope of balancing of interrupts.
The same information can be specified in the
INTCTL_DRV_IGNORE
section of the configuration file.
NOTE:
Please refer to the
INTCTL_DRV_IGNORE
section of the configuration file for drivers that
are currently not supported while balancing
interrupts.
- -I intr_id
Used with
-M
option to specify the interrupt ID
of the interrupt to be moved.
- -M
Migrate an interrupt to a specified CPU.
This option
must be specified with
the
-H, -I, -c
options.
The
-M
option should be specified first in the command line
followed by the remaining options which can be
specified in any order.
- -o
This option can be used for overriding some existing parameters from the
configuration file or for specifying new parameters.
Override driver weight information
(-o drv:driver_name:weight)
Specify a new driver weight or override an existing driver weight
specified in the configuration file.
driver_name
is the driver name of an existing driver in the configuration file section
INTCTL_DRIVER_WEIGHTS
or a new driver name.
weight
is the interrupt load that the driver may generate.
NOTE:
All I/O card drivers present on the system but not specified in the
INTCTL_DRIVER_WEIGHTS
section of the configuration file will be assigned a default weight of 10.
Override the trigger for balancing of interrupts
(-o trig:balance_on_cpu:distribute_to_cpu)
Using this option, the user can override the configuration file setting
INTCTL_BALANCE_TRIGGER.
These values determine when the interrupts will be balanced and the
scope of balancing of interrupts.
balance_on_cpu
is the minimum percentage of available number of CPUs that should be
handling interrupts.
Balancing of interrupts will start only if the number of CPUs
handling the interrupts is less than this percentage.
A value 100 will always trigger balancing of interrupts,
but if the system is optimally balanced with respect to interrupt
distribution then it may not result in any interrupt migrations.
The default value is 50.
distribute_to_cpu
is the percentage of available number of CPUs that should be handling
interrupts.
Balancing of interrupts would distribute interrupts across this
percentage of available number of CPUs.
The default value is 75.
NOTE:
If WLM (Work Load Manager) is configured to load balance across partitions
by migrating CPUs, it is desirable not to set this value to more than 75.
- -p
Display interrupt information about all the
CPUs on the system in a long format with spacing
in between the fields.
- -r file
Restore the system interrupt configuration from
the specified file,
file.
The interrupt configuration
is restored only if all the interface cards and
CPUs referenced in the saved configuration file
are still present on the system and the CPUs are
in the same state as in the saved configuration.
If new cards and new CPUs are added to the system,
intctl
will continue to restore the interrupt configuration
as long as the old configuration has not changed.
intctl
will fail to restore the interrupt configuration if
the file permission is not 0600.
In restoring the system configuration, the
command will assign interrupts from the
interface cards to the CPUs as specified in the
file.
- -s file
Save the system interrupt configuration to
the specified file,
file,
with file permission 0600.
If the file exists, the content of the
file will be overwritten and the file permissions
will be changed to 0600.
The command will store the interrupt information of all the CPUs on the system.
This file can be used to restore the
interrupt configuration of the system later using the
-r
option.
- -w
Force migrations without asking for user input.
Without this option, a user confirmation is requested while
balancing interrupts (option
-b).
When used with
-b,
-w
allows the user to balance interrupts without asking for
migrations that should be skipped.
Interrupt Configuration Display
The interrupt configuration can be displayed sorted by CPU ID
(intctl -p)
or
sorted by interface card hardware path
(intctl -H
hw_path).
By default, the command displays interrupt information about all
the interface cards on the system.
Here is a sample interrupt configuration display,
and the fields are explained below.
hw path class drv card cpu cpu intr intr card
name cell ID cell type ID description
=======================================================================
0/0/0/0 lan btlan N/A 0 N/A L 5 HP PCI
10/100Base-TX Core
0/0/1/0 ext_bus c720 N/A 0 N/A L 0 SCSI C895 Fast
Wide LVD
0/0/2/0 ext_bus c720 N/A 0 N/A L 1 SCSI C87x Ultra
Wide Single-Ended
0/0/2/1 ext_bus c720 N/A 0 N/A L 2 SCSI C87x Ultra
Wide Single-Ended
- hw path
A numerical string of hardware components
separated by slash
(/),
to represent a bus
converter.
The first component in the hardware path
is the cell (for a cell based system) or the
system bus adapter (for a non-cell based
system).
The system bus adapter is followed by
the address of the local bus adapter and the
interface card. Subsequent numbers are
separated by periods
(.).
Each number
represents the location of a hardware component
on the path of the device.
- class
The class of the interface
card, such as:
lan,
tty,
ext_bus.
- drv name
The driver associated
with the card.
- card cell
The cell number of the cell to which the card is connected.
- cpu ID
An integer value representing the identity of the
CPU to which the card's interrupt is assigned.
- cpu cell
The cell number of the cell to which the CPU is connected.
- intr type
A character representing the interrupt type:
- L
line based interrupt
- T
transaction based interrupt
- intr ID
The identity of the interrupt to be moved.
- card description
A brief description of the interface card.
- cpu_path
The hardware path of the CPU.
- cpu_state
Integer value representing the state of the
CPU: ENABLED(0), DISABLED(1) or RESERVED(2).
These states are interrupt states and do not
have any relationship to the thread state.
- ENABLED
The CPU is capable of receiving external interrupts from interface cards.
- DISABLED
The CPU cannot handle external
interrupts from interface cards.
- RESERVED
The state is reserved to receive
interrupts from specific cards,
for example, for RTE (Real Time Extensions) some processors are
reserved specifically to handle
interrupts from RTE cards.
NOTE:
N/A
value may be returned in some fields if the value is not available or the
feature not applicable.
The following is the output of
intctl -b
(balancing interrupts).
The following interrupt migrations will be performed for balancing interrupts
(algorithm selected:
driver_weight)
_____________________________________________________________________
Sl. Card Driver Intr From CPU ID To CPU ID
Num. H/W Path name ID (CPU H/W path) (CPU H/W path)
_____________________________________________________________________
1 1/0/8/1/1 fcd 1 0 (1/120) 7 (3/123)
2 1/0/8/1/0 fcd 1 0 (1/120) 6 (3/122)
3 1/0/0/1/0 igelan 1 0 (1/120) 5 (3/121)
4 1/0/0/3/0 c8xx 2 0 (1/120) 4 (3/120)
5 1/0/0/3/0 c8xx 1 0 (1/120) 4 (3/120)
6 1/0/0/2/1 c8xx 2 0 (1/120) 4 (3/120)
7 1/0/0/2/1 c8xx 1 0 (1/120) 4 (3/120)
Please select the migrations you want to skip.
Comma separated serial numbers or 'all' or 'none' : 1,2,7
Following migrations (serial number(s)) will be skipped.
1 2 7
Do you wish to skip these migrations (y/n):y
intctl: Moved the interrupt: 1, of card 1/0/0/1/0, driver igelan, from CPU:0 to CPU:5
intctl: Moved the interrupt: 2, of card 1/0/0/3/0, driver c8xx, from CPU:0 to CPU:4
intctl: Moved the interrupt: 1, of card 1/0/0/3/0, driver c8xx, from CPU:0 to CPU:4
intctl: Moved the interrupt: 2, of card 1/0/0/2/1, driver c8xx, from CPU:0 to CPU:4
Balancing of interrupts done.
The following is a description of each of the columns in the table:
- Sl. Num.
Serial number (starting from 1) of all the migrations that will be
performed for balancing of interrupts.
These serial numbers can be
used in selecting migrations that have to be skipped.
- Card H/W Path
The hardware path of I/O cards whose interrupt is getting migrated.
- Driver Name.
The driver associated with the card.
- Intr ID.
The identity of the interrupt to be moved.
- From CPU ID (CPU H/W path)
The CPU id and CPU hardware path to which the interrupt is currently
bound and the interrupt will be migrated off this CPU.
- To CPU ID (CPU H/W path)
The CPU id and CPU hardware path to which the interrupt will get
migrated.
Redirection
The
intctl
command allows the performance specialist to modify the
interrupt assignment of an interface card.
The user must specify the hardware path of interface card, the interrupt
ID that needs to be moved, and the new CPU ID that the interrupt will be
routed to.
When an interrupt is moved from one CPU to another, if the
interrupt shares a line with other interrupts, all the
interrupts on that line will be moved to the specified CPU.
The kernel will add a message to the
/var/adm/syslog/syslog.log
file which will contain the hardware path and interrupt IDs of
the interrupts being moved and the CPU ID of the CPU to which
these interrupts were moved.
When migrating an interrupt from one CPU to another, if the card
that the interrupt belongs to is in an erroneous or timed out
state, the interrupt will not be moved. If an interrupt shares a
line with other interrupts, and if any of the cards is in an
erroneous state, then none of the interrupts on the line will be
moved to the specified CPU.
Saving and Restoring System Interrupt Configurations
The
intctl
command can save and restore the system interrupt
configuration in a user specified file.
Before restoring the configuration, the
intctl
command checks to see if the system
setup has changed by checking that all the interface cards and
CPUs from the saved configuration are still present in the system
and that the CPUs are in the same state as in the saved
configuration.
The command will continue to restore the
configuration if new cards or CPUs have been added to the system
since the interrupt configuration was saved.
Interrupt Configuration File
/etc/intctl.conf
is the interrupt configuration file.
intctl
parameters can be saved in this configuration file, which makes them
persistent across reboot.
These parameters can be changed or overridden by the command line options
-i,
-o
and
-a.
The different sections in the
/etc/intctl.conf
configuration file described below.
- 1.
INTCTL_DRIVER_WEIGHTS
Each line after the above string is expected to be of the form
driver_name weight.
driver_name
is a string corresponding to the driver
and
weight
is an integer corresponding to the driver's weight.
These weights will be used while balancing interrupts using the
driver_weight
based algorithm.
If a driver is not specified in this section and is
present on the system, then a default weight of
10
is assumed. Weight can range from
0
to
INT_MAX
(see
limits(5)).
A
0
weight is considered as no interrupt load.
A positive integer is considered as the relative interrupt load
on the CPU with respect to different driver weights.
More weight corresponds to more interrupt load on the CPU.
The option
-o drv:driver_name:weight
can be used to override an existing driver weight or specify new
driver weights temporarily.
Example:
INTCTL_DRIVER_WEIGHTS
graf 300
- 2.
INTCTL_HW_IGNORE
Each line after the above string is expected to be of the form
hw_path intr_id.
hw_path
is a string corresponding to the hardware path of the I/O card and
intr_id
is an integer corresponding to the interrupt ID.
The specified
I/O card and the interrupt ID combination is ignored (that is,
will not be migrated) while balancing interrupts.
If interrupt ID is
-1,
then all the interrupt IDs associated with that I/O card are ignored.
The option
-i io:hw_path:intr_id
can be used to specify more I/O cards to be ignored temporarily.
NOTE: If an I/O card shares the interrupt line with another I/O card
whose driver is non MP-safe, then the interrupt of this I/O
card cannot be migrated.
intctl
will display the following message if this happens (the actual hardware
path will be different):
"NOTE: I/O card <hardware_path> is sharing the interrupt line
with another I/O card which is claimed by a non MP-safe driver.
Add the line 'hardware_path -1' (hardware_path intr_id) to
INTCTL_HW_IGNORE section of the configuration file /etc/intctl.conf
to avoid getting this message."
This entry needs to be made in
INTCTL_HW_IGNORE
section of the
/etc/intctl.conf
configuration file.
This will avoid selecting this card for further migrations
while balancing interrupts.
There is no impact on the system
or balancing of interrupts if this activity is not performed.
The only impact will be an interrupt migration failure message in
the syslog and the above
intctl
message.
Example:
INTCTL_HW_IGNORE
1/2/1/0 2
1/3/1/0 -1
- 3.
INTCTL_CPU_IGNORE
Each line after the above string is expected to be of the form
hw_path.
hw_path
is a string corresponding to the hardware path of the CPU that
should be ignored while balancing interrupts.
These CPUs will not be affected by balancing of interrupts.
The option
-i cpu:hw_path
can be used to ignore more CPUs temporarily.
Example:
- 4.
INTCTL_DRV_IGNORE
Each line after the above string is expected to be of the form
driver_name.
driver_name
is a string corresponding to the driver name of the driver that
should be ignored while balancing interrupts.
Any I/O card
claimed by this driver will not be selected for interrupt
migrations while balancing interrupts (irrespective of the
balancing algorithm chosen).
The option
-i drv:driver_name
can be used to ignore more drivers temporarily.
Example:
NOTE:
Please refer to this section of the configuration file for
drivers that are currently not supported while balancing
interrupts.
- 5.
INTCTL_DEFAULT_ALGO
The following line should specify the default algorithm to be used
while balancing interrupts (when
-a
is not used).
The only supported options are
driver_weight
and
round_robin.
The option
-a algorithm
can be used to change the algorithm temporarily.
Example:
INTCTL_DEFAULT_ALGO
driver_weight
- 6.
INTCTL_BALANCE_TRIGGER
The following line should specify the trigger for balancing of interrupts
and is expected to be of the form
balance_on_cpu distribute_to_cpu.
balance_on_cpu
is an integer that specifies the minimum percentage of number of
available CPUs that should be handling the interrupts.
distribute_to_cpu
is an integer that specifies the percentage of number of available
CPUs that can handle interrupts after balancing interrupts.
If the percentage of number of available CPUs handling the interrupts is
less than the
balance_on_cpu,
then balancing of interrupts is performed and interrupts are distributed across
distribute_to_cpu
percentage of CPUs.
Both values should be in the range 0-100 and
balance_on_cpu
should be smaller than
distribute_to_cpu.
NOTE:
If WLM (Work Load Manager) is configured to load balance across partitions by
migrating CPUs, it is desirable not to set this value to more than 75.
The option
-o trig :balance_on_cpu:distribute_to_cpu
can be used to override this setting temporarily.
Example:
INTCTL_BALANCE_TRIGGER
60 80
EXAMPLES
Display information about all interface cards which belong to the
class
lan:
Display the interrupt information of the card with hardware path
0/4/0/0/5/0:
Display interrupt information of all the interface cards under
the path,
0/4:
Display interrupt information of all interface cards under the
hardware path
0/4
and which belong to class
lan:
Display interrupt information about the CPU with CPU ID
3:
Migrate the interrupt with ID 1, coming from the card whose
hardware path is
0/4/0/0/5/0
to CPU
3:
intctl -M -H 0/4/0/0/5/0 -I 1 -c 3
Store the system interrupt configuration to
myconfig;
if
myconfig
already exists, its contents are overwritten:
Restore the system interrupt configuration from
myconfig:
Balance interrupts using
driver_weight
algorithm and without user confirmation:
intctl -b -w -a driver_weight
Balance interrupts only if less than 40% of the available
CPUs are handling interrupts, and distribute the interrupts
across 75% of the CPUs available:
intctl -b -w -o trig:40:75
Balance interrupts ignoring all interrupts of the I/O card
with hardware path
1/2/1/0,
ignore the CPU with hardware path
133/2,
and ignore the driver
graf1;
also, ask for confirmation before performing interrupt migrations:
intctl -b -i io:1/2/1/0:-1 -i cpu:133/2 -i drv:graf1
Balance interrupts according to the configuration file, but add a
new driver
graf
with weight 300 and change the weight of existing
btlan
driver from 10 (specified in the configuration file) to 15:
intctl -b -o drv:graf:300 -o drv:btlan:15
Balance interrupts if the current percentage of number
of available CPUs handling interrupts is below 60% and distribute
the interrupts across 80% of number of available CPUs the system:
