Overview

An interrupt is a hardware signal from a device to the CPU. It tells the CPU that the device needs attention, and the CPU should drop whatever it is doing and respond to the device. If the CPU is available (it is not doing something that is higher priority, such as servicing a higher priority interrupt) it suspends the current thread and eventually invokes the interrupt handler for that device. The job of the interrupt handler is to service the device, and stop it from interrupting. Once the handler returns, the CPU resumes whatever it was doing before the interrupt occurred.

The Solaris 2.x DDI/DKI provides a bus-architecture independent interface for registering and servicing interrupts. Drivers must register their device interrupts before they can receive and service interrupts.

Example

On x86 platforms, a device requests an interrupt by asserting an interrupt request line (IRQ) on the system bus. The bus implements multiple IRQ lines, and a particular device may be able to generate interrupts on one or more of them. Multiple devices may share a common IRQ line.

The bus IRQ lines are connected to an interrupt controller that arbitrates between interrupt requests. The kernel programs the interrupt controller to select which interrupts should be enabled at any particular time. When the interrupt controller determines that an interrupt should be delivered, it raises a request to the CPU. If processor interrupts are enabled, the CPU acknowledges the interrupt and causes the kernel to begin interrupt handler processing.

Interrupt Specification

An interrupt specification is the information the system needs in order to link an interrupt handler with a given interrupt. It describes the information provided by the hardware to the system when making an interrupt request. Interrupt specifications typically includes a bus-interrupt level. For vectored interrupts it includes an interrupt vector. On x86 platforms the driver.conf(4) file specifies the relative priority of the devices interrupt. See isa(4), eisa(4), mca(4), sbus(4), and vme(4) for specific information on interrupt specifications for these buses.

Interrupt Number

When registering interrupts the driver must provide the system with an interrupt number. This identifies which interrupt specification the driver is registering a handler for. Most devices have one interrupt, interrupt number zero. However, there are devices that have different interrupts for different events. A communications controller may have one interrupt for receive ready and one for transmit ready. The device driver normally knows how many interrupts the device has, but if the driver has to support several variations of a controller it can call ddi_dev_nintrs(9F) to find out the number of device interrupts. For a device with n interrupts, the interrupt numbers range from 0 to n-1.

Bus-Interrupt Levels

Buses prioritize device interrupts at one of several bus-interrupt levels. These bus interrupt levels are then associated with different processor-interrupt levels. For example, SBus devices that interrupt at SBus level 7 interrupt at SPARC level 9 on SPARCstation 2 systems, but interrupt at SPARC level 13 on SPARCstation 10 systems.

High-Level Interrupts

A bus-interrupt level that maps to a CPU interrupt priority level above the scheduler priority level is called a high-level interrupt. High-level interrupts must be handled without using system services that manipulate threads. In particular, the only kernel routines that high-level interrupt handlers are allowed to call are:

• mutex_enter(9F) and mutex_exit(9F) on a mutex initialized with an interrupt block cookie associated with the high-level interrupt.
• ddi_trigger_softintr(9F).

A bus-interrupt level by itself does not determine whether a device interrupts at high-level: a given bus-interrupt level may map to a high-level interrupt on one platform, but map to an ordinary interrupt on another platform. The function ddi_intr_hilevel(9F), given an interrupt number, returns a value indicating whether the interrupt is high-level.

The driver can choose whether or not to support high-level interrupts, but it always has to check --it cannot assume that its interrupts are not high-level. For information on checking for high-level interrupts see "Registering Interrupts" on page 111.

Types of Interrupts

There are two common ways to request for an interrupt: vectored and polled. Both methods commonly specify a bus-interrupt priority level. Their only difference is that vectored devices also specify an interrupt vector, but polled devices do not.

Vectored Interrupts

Devices that use vectored interrupts are assigned an interrupt vector. This is a number that identifies that particular interrupt handler. This vector may be fixed, configurable (using jumpers or switches), or programmable. In the case of programmable devices, an interrupt device cookie is used to program the device interrupt vector. When the interrupt handler is registered, the kernel saves the vector in a table.

When the device interrupts, the system enters the interrupt acknowledge cycle, asking the interrupting device to identify itself. The device responds with its interrupt vector. The kernel then uses this vector to find the responsible interrupt handler.

The VMEbus supports vectored interrupts.

Polled Interrupts

In polled (or autovectored) devices, the only information the system has about a device interrupt is its bus-interrupt priority level. When a handler is registered, the system adds the handler to list of potential interrupt handlers for the bus-interrupt level. When an interrupt occurs, the system must determine which device, of all the devices at that level, actually interrupted. It does this by calling all the interrupt handlers for that bus-interrupt level until one of them claims the interrupt.

The SBus supports polled interrupts.

Software Interrupts

The Solaris 2.x DDI/DKI supports software interrupts, also known as soft interrupts. Soft interrupts are not initiated by a hardware device, they are initiated by software. Handlers for these interrupts must also be added to and removed from the system. Soft interrupt handlers run in interrupt context and therefore can be used to do many of the tasks that belong to an interrupt handler.

Commonly, hardware interrupt handlers are supposed to be very quick, since they may suspend other system activity while running (particularly in high-level interrupt handlers). For example, they may prevent lower-priority interrupts from occurring while they run. For this reason, hardware interrupt handlers should do the minimum amount of work needed to service the device.

Software interrupt handlers run at a lower priority than hardware interrupt handlers, so they can do more work without seriously impacting the performance of the system. Additionally, if the hardware interrupt handler is high-level, it is severely restricted in what it can do. In this case, it is a good idea to simply trigger a software interrupt in the high-level handler and put all possible processing in the lower-level software interrupt handler.

Software interrupt handlers must not assume that they have work to do when they run, since (like hardware interrupt handlers) they can run because some other driver triggered a soft interrupt. For this reason, the driver must indicate to the soft interrupt handler that it should do work before triggering the soft interrupt.

Registering Interrupts

Before a device can receive and service interrupts, it must register them with the system by calling ddi_add_intr(9F). This provides the system with a way to associate an interrupt handler with an interrupt specification. This interrupt handler is called when the device might have been responsible for the interrupt. It is the handlers responsibility to determine if it should handle the interrupt, and claim it if so.

The following steps are usually performed in attach(9E):

• Test for high-level interrupts. Call ddi_intr_hilevel(9F) to find out if the interrupt specification maps to a high-level interrupt. If it does, one possibility is to post a message to that effect and return DDI_FAILURE. Code Example 6-1 on page 110 does this.
• Add the interrupt.
• Initialize any associated mutexes. There is a potential race condition between adding the interrupt handler and initializing mutexes. The interrupt routine is eligible to be called as soon as ddi_add_intr(9F) returns, as another device might interrupt and cause the handler to be invoked. This may result in the interrupt routine being called before any mutexes have been initialized with the returned interrupt block cookie. If the interrupt routine acquires the mutex before it has been initialized, undefined behavior may result.

  The solution to this problem is to use ddi_add_intr(9F) to add an interrupt handler that never claims the interrupt. This allows the driver to get the interrupt block cookie for the interrupt, which it can then use to initialize any mutexes. Once the mutexes are initialized, the temporary interrupt handler can be removed, and the real one installed. nulldev(9F) can be used as the temporary handler, though it needs to be cast properly. See Code Example 6-1 for an example.

Code Example 6-1 attach(9E) routine with temporary interrupt handler

static int
xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
    struct xxstate *xsp;
    if (cmd != DDI_ATTACH)
        return (DDI_FAILURE);
    ...
    if (ddi_intr_hilevel(dip, inumber) != 0){
        cmn_err(CE_CONT,
             "xx: high-level interrupts are not supported\n");
        return (DDI_FAILURE);
    }
    /*
     * The interrupt routine will grab the mutex, so a null */
     * handler is required.
     */
    if (ddi_add_intr(dip, inumber, &xsp->iblock_cookie,
        NULL, (u_int (*)(caddr_t))nulldev, NULL) != DDI_SUCCESS){
        cmn_err(CE_WARN, "xx: cannot add interrupt handler.");
        return (DDI_FAILURE);
    }
    mutex_init(&xsp->mu, "xx mutex", MUTEX_DRIVER,
        (void *) xsp->iblock_cookie);
    ddi_remove_intr(dip, inumber, xsp->iblock_cookie);
    if (ddi_add_intr(dip, inumber, &xsp->iblock_cookie,
        &xsp->idevice_cookie, xxintr, (caddr_t)xsp) != DDI_SUCCESS){
        cmn_err(CE_WARN, "xx: cannot add interrupt handler.");
        goto failed;
    }
    cv_init(&xsp->cv, "xx cv", CV_DRIVER, NULL);
    return (DDI_SUCCESS);
failed:

remove interrupt handler if necessary, destroy mutex

    return (DDI_FAILURE);
}

Responsibilities of an Interrupt Handler

The interrupt handler has a set of responsibilities to perform. Some are required by the framework, and some are required by the device. All interrupt handlers are required to do the following:

1. Possibly reject the interrupt.

   The interrupt handler must first examine the device and determine if it has issued the interrupt. If it has not, the handler must return DDI_INTR_UNCLAIMED. This step allows the implementation of device polling: it tells the system whether this device, among a number of devices at the given interrupt priority level, has issued the interrupt.

2. Inform the device that it is being serviced.

   This is a device-specific operation, but is required for the majority of devices. For example, SBus devices are required to interrupt until the driver tells them to stop. This guarantees that all SBus devices interrupting at the same priority level will be serviced.

   Most vectored devices, on the other hand, stop interrupting after the bus interrupt acknowledge cycle; however, their internal state still indicates that they have interrupted but have not been serviced yet.

3. Perform any I/O request related processing.

   Devices interrupt for different reasons, such as transfer done or transfer error. This step may involve reading the device's data buffer, examining the device's error register, and setting the status field in a data structure accordingly.

   Interrupt dispatching and processing is relatively expensive. The following points apply to interrupt processing:

• Do only what absolutely requires interrupt context.
• Do any additional processing that could save another interrupt, for example, read the next data from the device.

4. Return DDI_INTR_CLAIMED.

Code Example 6-2 Interrupt routine

static u_int
xxintr(caddr_t arg)
{

struct xxstate *xsp = (struct xxstate *) arg;
u_char       status, temp;

/*
 * Claim or reject the interrupt.This example assumes
 * that the device's CSR includes this information.
 */
mutex_enter(&xsp->mu);
status = xsp->regp->csr;
if (!(status & INTERRUPTING)) {
    mutex_exit(&xsp->mu);
    return (DDI_INTR_UNCLAIMED);
}
/*
 * Inform the device that it is being serviced, and re-enable
 * interrupts. The example assumes that writing to the
 * CSR accomplishes this. The driver must ensure that this write
 * operation makes it to the device before the interrupt service
 * returns. For example, reading the CSR, if it does not result in
 * unwanted effects, can ensure this.
 */
xsp->regp->csr = CLEAR_INTERRUPT | ENABLE_INTERRUPTS;
temp = xsp->regp->csr;

perform any I/O related and synchronization processing

signal waiting threads (biodone(9F) or cv_signal(9F))

    mutex_exit(&xsp->mu);
    return (DDI_INTR_CLAIMED);
}

On an architecture that does not support vectored hardware interrupts, when the system detects an interrupt, it calls the driver interrupt handler function for each device that could have issued the interrupt. The interrupt handler must determine whether the device it handles issued an interrupt. On architectures supporting vectored interrupts, this step is unnecessary but not harmful, and it enhances portability. The syntax and semantics of the interrupt handling routine therefore can be the same for both vectored interrupts and polling interrupts.

In the model presented here, the argument passed to xxintr() is a pointer to the state structure for the device that may have issued the interrupt. This was be set up by passing a pointer to the state structure as the intr_handler_arg argument to ddi_add_intr(9F) in attach(9E)

Most of the steps performed by the interrupt routine depend on the specifics of the device itself. Consult the hardware manual for the device to learn how to determine the cause of the interrupt, detect error conditions, and access the device data registers.

State Structure

This section adds the following fields to the state structure. See "State Structure" on page 57 for more information.

ddi_iblock_cookie_t       high_iblock_cookie;
ddi_idevice_cookie_t      high_idevice_cookie;
kmutex_t                  high_mu;
int                       softint_running;
ddi_iblock_cookie_t       low_iblock_cookie;
kmutex_t                  low_mu;
ddi_softintr_t            id;

Handling High-Level Interrupts

High-level interrupts are those that interrupt at the level of the scheduler and above. This level does not allow the scheduler to run, therefore high-level interrupt handlers cannot be preempted by the scheduler, nor can they rely on the scheduler (cannot block)--they can only use mutual exclusion locks for locking.

Because of this, the driver must use ddi_intr_hilevel(9F) to determine if it uses high-level interrupts. If ddi_intr_hilevel(9F) returns true, the driver can fail to attach, or it can use a two-level scheme to handle them. Properly handling high-level interrupts is the preferred solution.

---

Note - By writing the driver as if it always uses high level interrupts, a separate case can be avoided. However, this does result in an extra (software) interrupt for each hardware interrupt.

---

The suggested method is to add a high-level interrupt handler, which just triggers a lower-priority software interrupt to handle the device. The driver should allow more concurrency by using a separate mutex for protecting data from the high-level handler.

High-level Mutexes

A mutex initialized with the interrupt block cookie that represents a high-level interrupt is known as a high-level mutex. While holding a high-level mutex, the driver is subject to the same restrictions as a high-level interrupt handler. The only routines it can call are:

• mutex_exit(9F) to release the high-level mutex.
• ddi_trigger_softintr(9F) to trigger a soft interrupt.

Example

In the model presented here, the high-level mutex (xsp->high_mu) is only used to protect data shared between the high-level interrupt handler and the soft interrupt handler. This includes a queue that the high-level interrupt handler appends data to (and the low-level handler removes data from), and a flag that indicates the low-level handler is running. A separate low-level mutex (xsp->low_mu) is used to protect the rest of the driver from the soft interrupt handler.

Code Example 6-3 attach(9E) routine handling high-level interrupts

static int
xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
    struct xxstate *xsp;
    ...
    if (ddi_intr_hilevel(dip, inumber)) {
        /* add null high-level handler */
        if (ddi_add_intr(dip, inumber, &xsp->high_iblock_cookie,
             NULL, (u_int (*)(caddr_t))nulldev, NULL) != DDI_SUCCESS)
             goto failed;
        mutex_init(&xsp->high_mu, "xx high mutex", MUTEX_DRIVER,
             (void *)xsp->high_iblock_cookie);
        ddi_remove_intr(dip, inumber, xsp->high_iblock_cookie);
        if (ddi_add_intr(dip, inumber, &xsp->high_iblock_cookie,
             &xsp->high_idevice_cookie, xxhighintr, (caddr_t) xsp)
             != DDI_SUCCESS)
             goto failed;

        /* add null low-level handler */
        if (ddi_add_softintr(dip, DDI_SOFTINT_HI, &xsp->id,
             &xsp->low_iblock_cookie, NULL,
             (u_int (*)(caddr_t))nulldev, NULL))
             != DDI_SUCCESS)
             goto failed;
        mutex_init(&xsp->low_mu, "xx low mutex", MUTEX_DRIVER,
             (void *) xsp->low_iblock_cookie);
        ddi_remove_softintr(xsp->id);
        if (ddi_add_softintr(dip, DDI_SOFTINT_HI, &xsp->id,
             &xsp->low_iblock_cookie, NULL,
             xxlowintr, (caddr_t)xsp)) != DDI_SUCCESS)
             goto failed;
    } else {
        add normal interrupt handler
    }
    cv_init(&xsp->cv, "xx condvar", CV_DRIVER, NULL);
    ...
    return (DDI_SUCCESS);
failed:
    free allocated resources, remove interrupt handlers
    return (DDI_FAILURE);
}

The high-level interrupt routine services the device, and enqueues the data. The high-level routine triggers a software interrupt if the low-level routine is not running.

Code Example 6-4 High-level interrupt routine

static u_int
xxhighintr(caddr_t arg)
{
    struct xxstate *xsp = (struct xxstate *)arg;
    u_char status, temp;
    int     need_softint;

    mutex_enter(&xsp->high_mu);
    /* read status */
    status = xsp->regp->csr;

if (!(status & INTERRUPTING)) {
    mutex_exit(&xsp->high_mu);
    return (DDI_INTR_UNCLAIMED); /* device isn't interrupting */
}
xsp->regp->csr = CLEAR_INTERRUPT;
/* Flush store buffers */
temp = xsp->regp->csr;

read data from device and queue the data for the low-level interrupt handler;

    if (xsp->softint_running)
        need_softint = 0;
    else
        need_softint = 1;
    mutex_exit(&xsp->high_mutex);
    /* read-only access to xsp->id, no mutex needed */
    if (need_softint)
        ddi_trigger_softintr(xsp->id);
    return (DDI_INTR_CLAIMED);
}

The low-level interrupt routine is started by the high-level interrupt routine triggering a software interrupt. Once running, it should continue to do so until there is nothing left to process.

Code Example 6-5 Low-level interrupt routine

static u_int
xxlowintr(caddr_t arg)
{
    struct xxstate *xsp = (struct xxstate *) arg;
    ....
    mutex_enter(&xsp->low_mu);
    mutex_enter(&xsp->high_mu);
    if ( queue empty || xsp->softint_running ) {
        mutex_exit(&xsp->high_mu);
        mutex_exit(&xsp->low_mu);
        return (DDI_INTR_UNCLAIMED);
    }
    xsp->softint_running = 1;
    while ( data on queue ) {
        ASSERT(mutex_owned(&xsp->high_mu);
        dequeue data from high level queue;
        mutex_exit(&xsp->high_mu);

normal interrupt processing

        mutex_enter(&xsp->high_mu);
    }
    xsp->softint_running = 0;
    mutex_exit(&xsp->high_mu);
    mutex_exit(&xsp->low_mu);
    return (DDI_INTR_CLAIMED);
}