Windows Platform Design Notes

Design Information for the Microsoft ® Windows® Family of Operating Systems

Cancel Logic in Windows Drivers

Abstract
This paper provides information about writing drivers for the Microsoft ® Windows®
family of operating systems. It provides guidelines for driver writers to determine
when support for IRP cancellation is required and how to implement it properly.
Contents
Introduction..................................................................................................................................................3
When Must a Driver Support IRP Cancellation?.........................................................................................3
Cancel Logic with the Cancel Spin Lock.....................................................................................................3
Cancel Logic with a Driver-Supplied Lock...................................................................................................7
Cancel-Safe IRP Queues..........................................................................................................................11
How Cancel-Safe IRP Queues Work...................................................................................................12
Availability of Cancel-Safe IRP Queuing Routines..............................................................................18
Cancel Logic with StartIo, IoStartPacket, and IoStartNextPacket.............................................................18
Tips for Writing Cancel Routines...............................................................................................................20
Cancel Routines in Drivers that Hold IRPs Indefinitely........................................................................20
Cancel Routines that Search for IRPs in Queues................................................................................21
Avoiding Race Conditions..........................................................................................................................21
Testing for Errors.......................................................................................................................................24
Call to Action and Resources....................................................................................................................24
 Cancel Logic in Windows Drivers - 2

The information contained in this document represents the current view of Microsoft
Corporation on the issues discussed as of the date of publication. Because
Microsoft must respond to changing market conditions, it should not be interpreted
to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the
accuracy of any information presented after the date of publication.
This White Paper is for informational purposes only. MICROSOFT MAKES NO
WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE
INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user.
Without limiting the rights under copyright, no part of this document may be
reproduced, stored in or introduced into a retrieval system, or transmitted in any
form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of Microsoft
Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other
intellectual property rights covering subject matter in this document. Except as
expressly provided in any written license agreement from Microsoft, the furnishing
of this document does not give you any license to these patents, trademarks,
copyrights, or other intellectual property.

 2003 Microsoft Corporation. All rights reserved.

Microsoft, Windows, and Windows NT are either registered trademarks or
trademarks of Microsoft Corporation in the United States and/or other countries.
The names of actual companies and products mentioned herein may be the
trademarks of their respective owners.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 3

Introduction
Because I/O requests can be canceled asynchronously, IRP cancellation is difficult
to handle correctly in all cases. Cancel logic problems can remain dormant for long
periods of time because the code paths are rarely exercised. This paper describes
IRP cancellation in several contexts and shows the correct techniques for canceling
I/O requests.
The information in this paper applies to drivers for the Microsoft ® Windows® family
of operating systems. Driver writers can use this information to prevent driver errors
that are caused by faulty cancel logic.

When Must a Driver Support IRP Cancellation?

A driver must support IRP cancellation if one or both of the following statements are
true:

 The IRP can be queued indefinitely.

 The IRP is active on a device for long time or indefinitely.
If a driver supports cancellation, it can remove and quickly complete an outstanding
I/O request that a user has canceled. Because IRP cancellation occurs
asynchronously, drivers must be able to handle cancellation at any time while
processing an IRP.
Drivers can support IRP cancellation in any of the following ways:

 Use the system-wide cancel spin lock and include a Cancel routine.
 Use a driver-supplied locking mechanism and include a Cancel routine.
 Use the cancel-safe IRP queuing mechanism (IoCsqXxx). A Cancel routine
is not required.
All new drivers should use the IoCsqXxx routines, and existing drivers should be
revised to use the IoCsqXxx routines whenever possible. This paper provides
information on all three methods for the benefit of driver writers who maintain
existing drivers for which revising the cancellation logic is not feasible.
Drivers that merely forward IRPs to a lower-level driver must neither include a
Cancel routine nor use the IoCsqXxx mechanism. The lower-level driver is
responsible for canceling the IRP.

Cancel Logic with the Cancel Spin Lock

When a caller requests the cancellation of an IRP, the I/O Manager:
1. Acquires the cancel spin lock.
2. Sets the Cancel flag in the IRP to TRUE.
3. Sets the Cancel routine (if one exists) to NULL in an interlocked exchange.
4. Calls the Cancel routine if one was previously set.
The operating system provides a single cancel spin lock. While a thread holds the
cancel spin lock, only that thread can change the cancelable state of the IRP. For
example, while the caller holds the lock, the I/O Manager cannot call the Cancel
routine for that IRP. Likewise, another driver routine, such as a DispatchCleanup
routine, cannot simultaneously try to change the cancelable state of that IRP.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 4

The I/O Manager always holds the cancel spin lock when it calls a driver’s Cancel
routine. Consequently, if the Cancel routine acquires a second lock, a deadlock
could occur. For example, such a deadlock occurs when another thread within the
driver acquires a driver-created lock and then attempts to acquire the cancel spin
lock.
For drivers that perform I/O infrequently, use of the single system-wide cancel spin
lock is generally adequate. Drivers that perform I/O frequently, however, should not
use this technique. The single spin lock may become a bottleneck, thus causing
driver throughput to suffer, particularly on multiprocessor systems. Instead, such
drivers should use driver-supplied locks and the cancel-safe IRP queuing routines,
which are described in “Cancel-Safe IRP Queues.”
A driver that uses the cancel spin lock must follow these guidelines:

 Mark the IRP pending while holding the cancel spin lock.
 Complete the IRP with STATUS_CANCELLED if Irp->Cancel is set.
 Dequeue the IRP while holding the cancel spin lock.
 In the Cancel routine, remove the IRP from the pending structure while
holding the cancel spin lock.
 Do not acquire another lock while holding the cancel spin lock. After the
driver releases the cancel spin lock, it can acquire another lock if necessary.
By following these guidelines, a driver can avoid the race conditions that often
cause problems during IRP cancellation.

Example
The following sections show sample code that holds the cancel spin lock while it
queues a pending IRP, removes the IRP from the queue, and cancels the IRP. The
sample code demonstrates the correct techniques to avoid race conditions and
deadlocks. However, some error-handling code has been removed for brevity. The
circled numbers in the sample code correspond to the numbered lists that follow the
samples.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 5

Code to Queue a Pending IRP

The following sample code queues a pending IRP.
 IoAcquireCancelSpinLock (&oldirq);
 If (Irp->Cancel) {
 status = STATUS_CANCELLED;
// IRP should be completed after releasing the
// cancel spin lock.
} else {
 IoMarkIrpPending (Irp);
 IoSetCancelRoutine (Irp, CancelRoutine);
 PendIrpInStructure ();
status = STATUS_PENDING;
// IRP should not be accessed after the lock is
// released. The IRP could be grabbed by another
// thread or the Cancel routine could start.
}
 IoReleaseCancelSpinLock (oldirq);

if (status != STATUS_PENDING) {
Irp->IoStatus.Status = status;
 IoCompleteRequest( Irp, IO_NO_INCREMENT );
}

The following notes refer to the circled numbers in the sample code:
1. Acquire the cancel spin lock to protect the pending structure. While this thread
holds the cancel spin lock, no other thread can begin IRP cancellation or modify
the pending structure.
2. Check the Cancel flag in the IRP to determine whether the IRP was canceled
before this thread acquired the cancel spin lock.
3. The IRP was canceled already, so release the cancel spin lock, and then
complete the IRP.
4. The IRP has not been canceled, so mark it pending.
5. Set the Cancel routine. While the current thread holds the cancel spin lock, no
other thread can cancel the IRP. Note that the sample code tests the Cancel
flag in step 2, before it sets the Cancel routine. If the sample code instead set
the Cancel routine before it tested the flag, and if the IRP was already canceled,
it might be necessary to remove the Cancel routine.
6. Queue the IRP by using a local routine (not shown) named PendIrpInStructure.
Set the status to STATUS_PENDING to signal to subsequent code not to
complete the IRP.
7. Release the cancel spin lock.
8. Complete the IRP if it was already canceled in step 2.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 6

Code to Dequeue and Complete an IRP

The following sample code removes the IRP from the pending structure.
 … //Code has been removed for brevity.
 IoAcquireCancelSpinLock (&oldirq);
 Irp = DequeueIrpFromStructures ();
If (Irp) {
 IoSetCancelRoutine (Irp, NULL)) {
}
 IoReleaseCancelSpinLock (oldirq);
…//Code to do I/O has been removed for brevity
if (Irp) {
Irp->IoStatus.Status = status;
 IoCompleteRequest( Irp, IO_NO_INCREMENT );
}

The following notes refer to the circled numbers in the sample code:
1. If the IRP is canceled before this thread acquires the spin lock, the dequeuing
code does not handle the IRP. Instead, the Cancel routine removes the IRP
from the pending structure. See “Code for the Cancel Routine” for more
information.
2. Acquire the cancel spin lock to block IRP cancellation. This lock prevents the
Cancel routine for the IRP from running. The code does not need to check
whether the Cancel routine is already running; if the Cancel routine has already
run, it has already released the cancel spin lock and therefore has already
removed the IRP.
3. Remove the IRP from the structure.
4. Remove the Cancel routine for the IRP. Because this thread holds the cancel
spin lock, the Cancel routine cannot be running.
5. Release the cancel spin lock. The IRP can still be canceled after this point, but
because the Cancel routine has been removed, the I/O Manager sets Irp-
>Cancel to TRUE and does not call a Cancel routine.
6. Complete the IRP.

Code for the Cancel Routine

The following sample code cancels the IRP.
 RemoveFromPendingStructure (Irp);
 IoReleaseCancelSpinLock ();
Irp->IoStatus.Status = STATUS_CANCELLED;
 IoCompleteRequest(Irp, IO_NO_INCREMENT );

The following notes refer to the circled numbers in the sample code:
1. Remove the IRP from the pending structure before releasing the cancel spin
lock to prevent any other thread from trying to dequeue it.
2. Release the cancel spin lock. The I/O Manager acquires the cancel spin lock
before it calls the Cancel routine; the Cancel routine must release the lock.
3. Complete the IRP. A driver must never call IoCompleteRequest while it holds
the cancel spin lock.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 7

Cancel Logic with a Driver-Supplied Lock

Drivers that frequently perform I/O should not use the single, system-wide cancel
spin lock. Instead, they should use driver-supplied locks.
New drivers should use the cancel-safe IRP queuing mechanism described in
“Cancel-Safe IRP Queues” instead of the techniques described in this section. In
addition, Microsoft strongly recommends that existing drivers be revised to use
cancel-safe IRP queues instead of the Cancel routine techniques.
Code that uses a driver-supplied lock with a Cancel routine to cancel IRPs can be
challenging to write because of possible race conditions. The sample code in this
section shows how to correctly queue, dequeue, and cancel IRPs without using the
cancel spin lock. For details about possible race conditions, see “Avoiding Race
Conditions.”
Drivers that cancel IRPs without using the cancel spin lock must:

 Protect structures with a lock.

 Set and clear the Cancel routine while holding the lock.
 If the Cancel routine starts to run while the driver dequeues and processes
the IRP, allow the Cancel routine to complete the IRP.
 Hold the lock that protects the pending structure in the Cancel routine.

Example
The following three sections show sample code that queues a pending IRP,
removes the IRP from the queue, and cancels the IRP, using a driver-supplied lock.
The sample code demonstrates the correct techniques to avoid race conditions and
deadlocks. However, some error-handling code has been removed for brevity.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 8

Code to Queue an IRP

The following sample code marks an IRP pending and assigns it to a structure
(such as a queue, a global location, or a pointer in the driver extension). If the IRP is
canceled, the Cancel routine removes the IRP from the structure.
 KeAcquireSpinLock(&devExtension->QueueLock, &oldIrql);
IoSetCancelRoutine (Irp, CsampCancelIrp);

 if (Irp->Cancel &&
 IoSetCancelRoutine (Irp, NULL)) {
//
// The IRP has already been canceled, but the I/O
// Manager has not yet called the Cancel routine.
 status = STATUS_CANCELLED;
//
// Complete the IRP later, after releasing the lock.
//
}
else {
 IoMarkIrpPending(Irp);
 InsertTailList(&devExtension->PendingIrpQueue,
&Irp->Tail.Overlay.ListEntry);
status = STATUS_PENDING;
}
}

 KeReleaseSpinLock(&devExtension->QueueLock, oldIrql);

The following notes refer to the circled numbers in the sample code:
1. Obtain a lock and then set the Cancel routine. If the driver does not hold a lock,
the Cancel routine could run to completion before the next line of code is
executed, thus invalidating the reference to the IRP in the call to
IoSetCancelRoutine.
2. Test the Cancel flag to determine whether the IRP has already been canceled.
The IRP could have been canceled before the Cancel routine was set or
afterwards; point 3 determines when the cancellation occurred.
While the sample code holds the lock, the Cancel routine cannot run. If the
routine has already started to run, it is blocked while it waits for the lock.
Consequently, the IRP remains valid.
3. Determine whether the Cancel routine has started running. If not, remove the
routine to prevent it from starting.
The logical AND in the if statement is evaluated from left to right, and evaluation
stops as soon as the value is determined. Therefore:
 If Irp->Cancel is FALSE, the IRP has not been canceled. The test
stops and execution continues with the else clause.
 If Irp->Cancel is TRUE, the call to IoSetCancelRoutine is
executed, which removes the Cancel routine.
If IoSetCancelRoutine returns a valid pointer (TRUE), the IRP has been
canceled, but the Cancel routine is not running, so execution continues with
point 4.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 9

If IoSetCancelRoutine returns NULL (FALSE), the Cancel routine has

already started, so execution continues with the else clause. The Cancel
routine will dequeue and complete the IRP, so mark the IRP pending.
The interlocked sequence in IoSetCancelRoutine ensures that the value of Irp-
>Cancel is read before the Cancel routine is removed. If Irp->Cancel is not
true, the example code cannot miss a cancellation event.
4. Because no Cancel routine is running for this IRP, release the lock and then
complete the IRP in this routine.
5. The IRP has not been canceled. Mark it pending.
6. Save the pending IRP in the structure where other threads can find it.
7. Release the lock.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 10

Code to Dequeue and Complete an IRP

The following code removes the IRP from the structure. If the IRP has been
canceled, the code completes it; if not, the code marks the IRP as the current IRP
for the device.
The following notes refer to the circled numbers in the sample code:
 nextIrp = NULL;

 KeAcquireSpinLock(&devExtension->QueueLock, &oldIrql);

 while(!IsListEmpty(&devExtension->PendingIrpQueue))
{
listEntry =
RemoveHeadList(&devExtension->PendingIrpQueue);
nextIrp = CONTAINING_RECORD(listEntry, IRP,
Tail.Overlay.ListEntry);
 if (IoSetCancelRoutine (nextIrp, NULL))
{
//
// Cancel routine cannot run now and cannot already have
// started to run.
//
break;
}
else {
//
// The Cancel routine is running. Leave the IRP alone.
//
 InitializeListHead(listEntry);
nextIrp = NULL;
}
}

 KeReleaseSpinLock(&devExtension->QueueLock, oldIrql);

1. If the IRP is canceled before this thread obtains a lock, IRP cancellation can
proceed to any point in the Cancel routine.
2. If the IRP is canceled after this thread obtains the lock, the Cancel routine
blocks until it can acquire a lock.
3. Look for the IRP in the queue. If the IRP was canceled, and its Cancel routine
acquired a lock before the current thread, the Cancel routine has already
removed the IRP from the queue. Therefore, one of the following must be true
for all of the IRPs in the queue:
 The IRP has not been canceled.
 The IRP has been canceled, but its Cancel routine has blocked
before obtaining a lock.
4. Remove the Cancel routine and determine whether the cancel code is active. If
IoSetCancelRoutine returns a non-NULL value (TRUE), the Cancel routine has
not already started; it cannot start after the Cancel routine is removed because
removal is performed in an interlocked exchange.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 11

5. If IoSetCancelRoutine returns NULL (false), the Cancel routine is already

running. Reset the local IRP pointer; the Cancel routine removes the IRP from
the structure and completes it.
6. Release the lock.

Code for the Cancel Routine

The following is the corresponding Cancel routine. This routine dequeues canceled
IRPs and completes them.

 IoReleaseCancelSpinLock(Irp->CancelIrql);
//
// Acquire the queue lock
//
 KeAcquireSpinLock(&devExtension->QueueLock, &oldIrql);
//
// Remove the canceled IRP from the queue and
// release the queue lock.
//
 RemoveEntryList(&Irp->Tail.Overlay.ListEntry);
 KeReleaseSpinLock(&devExtension->QueueLock, oldIrql);
//
// Complete the request with STATUS_CANCELLED.
//
 Irp->IoStatus.Status = STATUS_CANCELLED;
Irp->IoStatus.Information = 0;
IoCompleteRequest (Irp, IO_NO_INCREMENT);

The following notes refer to the circled numbers in the sample code:
1. Release the cancel spin lock that the I/O Manager acquired.
2. Acquire the driver-created lock. If another thread is traversing the protected
structure, the Cancel routine blocks while it waits for the lock.
3. Remove the IRP from the pending structure.
4. Release the lock.
5. Complete the IRP unconditionally.

Cancel-Safe IRP Queues

Until the release of Windows XP, drivers that supported IRP cancellation without
using the cancel spin lock were required to include a Cancel routine as described in
the preceding section. Logic in such routines was often incorrect because of several
possible race conditions. Nevertheless, this method was recommended for drivers
that frequently performed I/O because of the performance limitations of the cancel
spin lock. With the release of the Windows XP DDK, drivers no longer must supply
a Cancel routine. Instead, they can use new cancel-safe IRP queuing routines.
The cancel-safe IRP queuing routines (the IoCsqXxx routines) provide a framework
to implement IRP queuing. The framework handles cancellation properly so that
race conditions do not occur. Drivers that use these routines do not implement
Cancel routines. Instead, they supply driver-specific callback routines that manage
queues and locks. While the callback routines are running, the I/O Manager ensures
that IRP cancellation code does not run.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 12

All new drivers for Windows 2000 and later releases should use the IoCsqXxx
routines. See “Availability of Cancel-Safe IRP Queuing Routines” for information on
compiling and linking drivers that use these routines.

How Cancel-Safe IRP Queues Work

Cancel-safe IRP queues are designed for drivers that do not use the cancel spin
lock. When a driver uses the cancel-safe IRP queuing routines, the I/O Manager
protects against the race conditions that can occur during IRP cancellation. To use
the cancel-safe IRP queuing mechanism, a driver must provide:

 Driver-specific callback routines:

 XxxCsqInsertIrp
 XxxCsqRemoveIrp
 XxxCsqPeekNextIrp
 XxxCsqAcquireLock
 XxxCsqReleaseLock
 XxxCsqCompleteCanceledIrp
 A lock with which to lock the queue.
 Storage for the queue of pending IRPs.
A driver that uses the cancel-safe IRP queuing routines should handle cancelable
IRPs as follows:
1. To queue the IRP, call IoCsqInsertIrp or IoCsqInsertIrpEx. A driver might
make this call from its StartIo routine.
In response, the I/O Manager calls the driver’s XxxCsqAcquireLock routine to
lock the queue and then calls the driver’s XxxCsqInsertIrp routine to insert the
IRP in the queue. IoCsqInsertIrp marks the IRP pending and returns context
information that identifies the IRP. The driver later passes the context
information to IoCsqRemoveIrp when it removes the IRP from the queue.
2. To dequeue an IRP, call IoCsqRemoveNextIrp or IoCsqRemoveIrp.
IoCsqRemoveIrp finds the IRP that matches the context information that was
returned when the driver queued the IRP. In response to IoCsqRemoveIrp, the
I/O Manager calls the driver’s XxxIoCsqRemoveIrp routine to remove the IRP
from the queue.
IoCsqRemoveNextIrp dequeues the next IRP that matches driver-specified
criteria. In response to IoCsqRemoveNextIrp, the I/O Manager calls the
driver’s XxxIoCsqPeekNextIrp to find the IRP, then calls the driver’s
XxxIoCsqRemoveIrp routine to remove the IRP from the queue.
When a user cancels an I/O request, the I/O Manager calls the driver’s
XxxIoCsqPeekNextIrp and XxxIoCsqRemoveIrp routines as necessary to find and
dequeue the IRP. After dequeuing the IRP, the I/O Manager calls the driver’s
XxxCsqCompleteCanceledIrp routine to cancel and complete the IRP.

Example
The sample code in the following sections shows the callback routines that a driver
must implement to use the cancel-safe IRP queuing mechanism.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 13

Code to Define and Initialize Structures

The driver allocates a queue lock and an IO_CSQ structure for the cancel-safe IRP
queue in its device extension, as follows:
typedef struct _DEVICE_EXTENSION{

// IRPs waiting to be processed are queued here.

LIST_ENTRY PendingIrpQueue;

// Spin lock to protect access to the queue.

KSPIN_LOCK QueueLock;

// Pointer to current device IRP. Exclusive access to

// this field is also provided by the QueueLock.
PIRP CurrentIrp;

// … Some driver-specific fields deleted for brevity.

// Cancel-safe IRP queue.

IO_CSQ CancelSafeQueue;

} DEVICE_EXTENSION, *PDEVICE_EXTENSION;

In its DriverEntry routine, the driver calls IoCsqInitialize or IoCsqInitializeEx to

initialize the dispatch table for the cancel-safe IRP queue. For example:
IoCsqInitializeEx( &devExtension->CancelSafeQueue,
CsampInsertIrp,
CsampRemoveIrp,
CsampPeekNextIrp,
CsampAcquireLock,
CsampReleaseLock,
CsampCompleteCanceledIrp );

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 14

Code to Queue an IRP

The following code is a sample XxxCsqInsertIrp routine. When the driver receives a
cancelable IRP, it calls IoCsqInsertIrp or IoCsqInsertIrpEx to queue the IRP. In
return, the I/O Manager acquires the necessary locks, manages possible race
conditions, and calls this routine to place an IRP in the cancel-safe queue.
NTSTATUS CsampInsertIrp (
IN PIO_CSQ Csq,
IN PIRP Irp,
IN PVOID InsertContext
)
{
PDEVICE_EXTENSION devExtension;

 devExtension = CONTAINING_RECORD(Csq,
DEVICE_EXTENSION, CancelSafeQueue);

if (!devExtension->CurrentIrp) {
devExtension->CurrentIrp = Irp;
return STATUS_UNSUCCESSFUL;
}

 InsertTailList(&devExtension->PendingIrpQueue,
&Irp->Tail.Overlay.ListEntry);
return STATUS_SUCCESS;
}
The following notes refer to the circled numbers in the sample code:
1. Get a pointer to the device extension. The driver has allocated the cancel-safe
queue in its device extension, as shown in “Code to Define and Initialize
Structures.”
2. Insert the IRP at the end of the queue.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 15

Code to Dequeue an IRP

When the driver is ready to process an IRP from the queue, it calls
IoCsqRemoveIrp to remove a specific IRP or calls IoCsqRemoveNextIrp to
remove the next IRP that matches specific criteria. In return, the I/O Manager
determines which IRP to remove, calling the driver’s XxxCsqPeekNextIrp routine if
necessary, then calls the driver’s XxxCsqRemoveIrp routine to remove the IRP from
the queue.
The following code is a sample XxxCsqRemoveIrp routine.
VOID CsampRemoveIrp(
IN PIO_CSQ Csq,
IN PIRP Irp
)
{
PDEVICE_EXTENSION devExtension;

 devExtension = CONTAINING_RECORD(Csq,
DEVICE_EXTENSION, CancelSafeQueue);
 RemoveEntryList(&Irp->Tail.Overlay.ListEntry);
}

The following notes refer to the circled numbers in the sample code:
1. Get a pointer to the device extension, which contains the cancel-safe queue.
2. Remove the IRP from the queue.

Code to Search for an IRP

When a driver calls IoCsqRemoveNextIrp, it passes driver-specified context
information that identifies the IRP to remove. In response, the I/O Manager calls the
driver’s XxxCsqPeekNextIrp routine. This routine allows a driver to search through
the cancel-safe IRP queue for the next IRP that meets driver-specified criteria.
The following is a sample XxxCsqPeekNextIrp routine.
PIRP CsampPeekNextIrp(
IN PIO_CSQ Csq,
IN PIRP Irp,
IN PVOID PeekContext
)
{
PDEVICE_EXTENSION devExtension;
PIRP nextIrp = NULL;
PLIST_ENTRY nextEntry;
PLIST_ENTRY listHead;
PIO_STACK_LOCATION irpStack;

devExtension = CONTAINING_RECORD(Csq,
DEVICE_EXTENSION, CancelSafeQueue);

listHead = &devExtension->PendingIrpQueue;

//
// If the IRP is NULL, we will start peeking from the
// head of the list. If not, we will start from the

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 16

// current IRP onwards. We assume that

// new IRPs are always inserted at the tail.
//

 if(Irp == NULL) {
nextEntry = listHead->Flink;
} else {
nextEntry = Irp->Tail.Overlay.ListEntry.Flink;
}

 while(nextEntry != listHead) {
nextIrp = CONTAINING_RECORD(nextEntry, IRP,
Tail.Overlay.ListEntry);

irpStack = IoGetCurrentIrpStackLocation(nextIrp);

//
// If context is present, continue until we find a
// matching IRP. If not, break out of the loop
// as soon as we get the next IRP.
//

 if(PeekContext) {
 if(irpStack->FileObject ==
(PFILE_OBJECT)PeekContext) {
break;
}
} else {
break;
}
nextIrp = NULL;
nextEntry = nextEntry->Flink;
}

//
// Check if this is from start packet.
//
 if (PeekContext == NULL) {
devExtension->CurrentIrp = nextIrp;
}

 return nextIrp;

The following notes refer to the circled numbers in the sample code:
1. Initialize a pointer to the list of IRPs.
2. Get the next IRP from the list.
3. When the driver called IoRemoveNextIrp, it passed a pointer to context
information that identifies the IRP to remove; in turn, the I/O Manager passes
that pointer to the XxxCsqPeekNextIrp routine. If the pointer is NULL, this
routine should return the next IRP in the list. If the pointer is not NULL, this
routine must walk the list until it finds an IRP with the matching context.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 17

4. This driver uses the FileObject from the IRP stack location as the context, so
that it can remove the next IRP for a particular file. Passing the FileObject as
the peek context is useful for handling IRP_MJ_CLEANUP requests. The peek
context is not required to be the FileObject; it can be any information that the
driver requires to determine which IRP to remove from the queue. For example,
a driver that handles IRPs based on some internal priority scheme might pass
information related to the priority.
5. This driver implements an internal start-packet routine to sequence the
processing of I/O requests. If XxxCsqPeekNextIrp is called without a context,
the routine sets the current IRP.
6. Return a pointer to the IRP.

Code to Acquire and Release Locks

The following sample code acquires a lock for the cancel-safe IRP queue. The
driver has previously allocated the lock in its DriverEntry routine. Although this
example uses a spin lock, a driver could also use a mutex or other synchronization
technique.
VOID CsampAcquireLock(
IN PIO_CSQ Csq,
OUT PKIRQL Irql
)
{
PDEVICE_EXTENSION devExtension;

 devExtension = CONTAINING_RECORD(Csq,
DEVICE_EXTENSION, CancelSafeQueue);
 KeAcquireSpinLock(&devExtension->QueueLock, Irql);
return;
}

The following notes refer to the circled numbers in the sample code:
1. Get a pointer to the device extension.
2. Acquire the spin lock, which is allocated in the device extension.
The following sample code releases the previously acquired lock:
VOID CsampReleaseLock(
IN PIO_CSQ Csq,
IN KIRQL Irql
)
{
PDEVICE_EXTENSION devExtension;

 devExtension = CONTAINING_RECORD(Csq,
DEVICE_EXTENSION, CancelSafeQueue);

 KeReleaseSpinLock(&devExtension->QueueLock, Irql);
}
The following notes refer to the circled numbers in the sample code:
1. Get a pointer to the device extension.
2. Release the spin lock.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 18

Code to Complete an IRP

After a user cancels an I/O request, the I/O Manager calls the driver’s
XxxCsqCompleteCanceledIrp routine with the IRP. The driver is not required to
dequeue the IRP; the I/O Manager handles this step.
The following code is a sample XxxCsqCompleteCanceledIrp routine:
VOID CsampCompleteCanceledIrp(
IN PIO_CSQ pCsq,
IN PIRP Irp
)
{
Irp->IoStatus.Status = STATUS_CANCELLED;
Irp->IoStatus.Information = 0;
IoCompleteRequest(Irp, IO_NO_INCREMENT);
}

The sample routine merely calls IoCompleteRequest. A

XxxCsqCompleteCanceledIrp routine might also perform other device-specific
cancellation tasks, if required.

Availability of Cancel-Safe IRP Queuing Routines

Support for the cancel-safe IRP queuing routines is available in the Windows XP
and later DDKs. These routines can be used in drivers for the Windows 2000 and
later operating systems.
The way you compile and link a driver with these routines varies, depending on the
DDK you are using and the target operating system for your driver. See the
Windows DDK documentation for details.

Cancel Logic with StartIo, IoStartPacket, and

IoStartNextPacket
If a driver passes a pointer to a Cancel routine in its call to IoStartPacket, the I/O
Manager sets the Cancel routine for the IRP and calls the driver’s StartIo routine
with the IRP in a cancelable state. The driver, in turn, should also call
IoStartNextPacket with the Cancelable parameter set to TRUE. The StartIo routine
must check for cancellation and protect against related race conditions.
Drivers for Windows XP and later versions can ensure that the StartIo routine is
never called with a cancelable IRP. If a driver calls IoSetStartIoAttributes with the
Noncancelable parameter set to TRUE, the I/O Manager removes cancellation from
an IRP before it calls the driver’s StartIo routine. Doing so greatly simplifies the
StartIo routine, because it is not required to handle IRP cancellation at all. The
driver should use the Cancelable parameter in subsequent calls to
IoStartNextPacket and IoStartNextPacketByKey. However, the driver still must
implement a Cancel routine to remove the IRP from the device queue. The I/O
Manager cannot cancel the IRP by itself, because the driver might have allocated
per-IRP resources that must be freed when the IRP is cancelled.
The use of IoStartPacket, IoStartNextPacket, and IoStartNextPacketByKey with
IRP cancellation is not recommended for drivers that frequently perform I/O,
because the I/O Manager holds the global cancel spin lock while it inserts and
removes IRPs from the device queue. Instead, drivers should use the cancel-safe
IRP queues to implement queues that have similar semantics to the device queue.
The example that follows shows such an implementation.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 19

Example
The following code samples show correct coding techniques for a StartIo routine
and the corresponding Cancel routine.

Code to Initialize the Device Queue

After the driver has created its device object (in its AddDevice or DriverEntry
routine), it should call IoSetStartIoAttributes to set the Noncancelable and
DeferredStartIo parameters to TRUE. The Noncancelable parameter ensures that
the IRP is not cancelable when the StartIo routine is called. The DeferredStartIo
parameter prevents recursive calls to the StartIo routine if IoStartNextPacket is
called within StartIo.
AddDevice(…)
{
IoCreateDevice(…, &deviceObject)
IoSetStartIoAttributes(deviceObject, TRUE, TRUE);
…
}

Code for the StartIo Routine

If a driver’s StartIo routine cannot be called with cancelable IRPs, the routine does
not include code to handle IRP cancellation. As a result, the StartIo routine is quite
straightforward. The following code shows such a StartIo routine:
VOID SioStartIo(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
)
{
NTSTATUS ntStatus = STATUS_PENDING;
//
// Perform the IO.
//
//…Code deleted for brevity.
//
//
// I/O is done. Set status and complete the IRP.
//

Irp->IoStatus.Status = ntStatus;

IoStartNextPacket(DeviceObject, TRUE);

IoCompleteRequest (Irp, IO_NO_INCREMENT);

return;
}

The StartIo routine performs the I/O, and then it calls IoStartNextPacket and
IoCompleteRequest to get the next IRP and complete the current IRP.
Because this driver used IoSetStartIoAttrbutes to disable recursive calls into its
StartIo routine, the StartIo routine can safely call IoStartNextPacket. By calling
IoStartNextPacket before IoCompleteRequest, the driver can keep the hardware
busy while the operating system completes the IRP.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 20

Code for the Cancel Routine

The following code is the Cancel routine that corresponds to the StartIo routine:
VOID
SioCancelIrp(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
)

{
BOOLEAN wasQueued;

//
// Remove the IRP from the queue.
//
 wasQueued =
KeRemoveEntryDeviceQueue(&DeviceObject->DeviceQueue,
&Irp->Tail.Overlay.DeviceQueueEntry );
IoReleaseCancelSpinLock( Irp->CancelIrql );

 if(wasQueued)
{
Irp->IoStatus.Status = STATUS_CANCELLED;
Irp->IoStatus.Information = 0;
IoCompleteRequest(Irp, IO_NO_INCREMENT);
}
else
{
// The IRP was not found in the queue.
// A serious problem has occurred.
ASSERT(FALSE);
}
return;
}

The following notes refer to the circled numbers in the sample code:
1. Remove the IRP from the queue and release the cancel spin lock.
2. Cancel and complete the IRP.

Tips for Writing Cancel Routines

The following sections describe techniques for writing Cancel routines in drivers that
either hold IRPs indefinitely or search for a particular IRP in a queue.

Cancel Routines in Drivers that Hold IRPs Indefinitely

In some drivers, IRP completion could take an indefinite amount of time. For
example, a read IRP that is queued to a serial port could be held indefinitely until a
user presses a key. Drivers should handle cancellation of such IRPs as follows:
1. Allocate memory for each IRP to hold the IRP context.
2. In the routine that queues the IRP, associate the IRP with the context, so that
the context points to the IRP and conversely.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 21

3. In the routine that dequeues the IRP, remove the association and free the
context.
4. In the Cancel routine, remove the association and complete the IRP, but do not
free the context.
5. When the deferred procedure call (DPC) or interrupt service routine (ISR) runs,
which indicates that the hardware received a character, the driver does not
check at the IRP. Instead, the driver checks the context and then determines
whether the IRP has been canceled. If the IRP has not been canceled, the
driver completes the IRP. If the IRP has been canceled, the driver frees the
context.

Cancel Routines that Search for IRPs in Queues

Some drivers implement a queue-searching algorithm in their Cancel routines.
Queue-searching techniques are easy to implement, and Cancel routines that use
them are not as difficult to write because they do not remove IRPs that they cannot
find in the queue. However, if the queues are large, searching a queue in a Cancel
routine can slow driver performance. In general, the amount of time required to
search for and cancel many IRPs is proportional to the square of the number of
IRPs being canceled.
A subtle race condition can occur in such Cancel routines if the searching code tries
to handle cases where another thread might already have completed the IRP. The
race condition occurs if the driver does not use the cancel spin lock to protect the
structures that hold the pending IRPs, but instead uses another type of lock. In this
situation, the Cancel routine releases the cancel spin lock and then searches for the
IRP to cancel. If the Cancel routine does not find the IRP, the IRP must have been
completed already by another thread. The subtle flaw with this approach is that
releasing the cancel spin lock requires the I/O Manager to access the IRP, because
the IRP contains the original IRQL for the request.
As a general rule, if structures are protected by something other than the cancel
spin lock, other threads must check the return value of IoSetCancelRoutine to
determine whether the Cancel routine is running (or is about to run) and must
always allow the Cancel routine to complete the IRP.

Avoiding Race Conditions

Drivers that set Cancel routines for IRPs include code to perform the following three
operations:

 Queue the IRP, set the Cancel routine, mark the IRP pending, and return a
pending status.
 Dequeue the IRP, remove its Cancel routine, and process the IRP (typically
to completion).
 Cancel the IRP by removing it from the queuing structure and completing it.

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 22

A race condition can occur if an IRP is canceled at any of the following points:

 After a driver routine is called, but before it queues the IRP. A race
condition can occur between the driver routine and the Cancel routine.
 After a driver routine is called, but before it starts to process an IRP. For
example, an IRP might be canceled after a driver's StartIo routine is called, but
before the StartIo routine removes the IRP from the device queue. In this case,
the race condition occurs between the StartIo routine and the Cancel routine.
 After the driver routine dequeues the IRP, but before it starts the requested
I/O. The race condition occurs between the dequeuing routine and the Cancel
routine.

Note: A race condition can also occur if the event that completes a pending IRP occurs
before the driver has finished queuing the IRP. This is not strictly a cancellation problem, but
it is similar.

After a driver queues an IRP and releases any spin locks that protect the queue,
another thread can access and change the IRP. When the original thread resumes
— possibly as soon as the next line of code — the IRP might have already been
canceled or otherwise changed.
Understanding which thread owns, or has access to, the IRP at each point in
processing can help find such problems. Examples 1 and 2 trace IRP ownership to
demonstrate the issues.

Example 1
In the following example, a race condition can occur between the sample code and
the Cancel routine (not shown) if the IRP is canceled after the driver releases the
cancel spin lock but before it marks the IRP pending.
case IOCTL_GET_REQUEST:

IoAcquireCancelSpinLock (&cancelIRQL);
InsertTailList (&ConnectionIrpQueue,
&Irp->Tail.Overlay.ListEntry);
IoSetCancelRoutine (Irp, GetReqCancel);
IoReleaseCancelSpinLock (cancelIRQL);

IoMarkIrpPending (Irp);

As soon as this thread stores the address of the IRP in the global structure
(ConnectionIrpQueue) and releases the cancel spin lock, another thread can take
ownership of the IRP. The new owning thread then acquires the cancel spin lock
and reads the queued IRP by traversing the global structures. By the time that the
original thread marks the IRP pending, the other thread might already have
completed the IRP.
To correct the problem, the driver must hold the cancel spin lock when it marks the
IRP pending and sets status. For example:
case IOCTL_GET_REQUEST:

IoAcquireCancelSpinLock (&cancelIRQL);
InsertTailList (&ConnectionIrpQueue,
&Irp->Tail.Overlay.ListEntry);
IoSetCancelRoutine (Irp, GetReqCancel);
IoMarkIrpPending (Irp);

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 23

Irp->IoStatus.Status = status = STATUS_PENDING;

IoReleaseCancelSpinLock (cancelIRQL);

Example 2
In the following example, a race condition can occur between the routine that
dequeues the IRP and the Cancel routine for the IRP.
if(!IsListEmpty( &pDevExt->IrpQ[Type].ListHead) ) {
KeAcquireSpinLock( &pDevExt->IrpQ[Type].SpinLock,
&kTempIrql );
pHead = RemoveHeadList( &pDevExt->IrpQ[Type].ListHead );
KeReleaseSpinLock( &pDevExt->IrpQ[Type].SpinLock,
kTempIrql );

pIrp = CONTAINING_RECORD(
pHead, // Address
IRP, // Type
Tail.Overlay.ListEntry ); // Field

pPreviousCancelRoutine = IoSetCancelRoutine( pIrp, NULL );

The sample code includes several errors. The first line incorrectly checks whether
the list of IRPs is empty before it obtains a lock to protect the list. If the IRP is
canceled after the test completes but before the driver acquires the spin lock, the
code within the lock could attempt to remove an IRP from an empty list.
Furthermore, the sample code releases the lock before it removes the Cancel
routine. Because the sample code does not use the cancel spin lock, no locks are in
place when it calls IoSetCancelRoutine. Therefore, if the IRP is canceled after the
call to KeReleaseSpinlock, the Cancel routine might already have started to run
when the sample code tries to remove the Cancel routine. To correct this error, the
driver should check the result of IoSetCancelRoutine. If IoSetCancelRoutine
returns NULL, IRP cancellation is already in progress.
The following code shows the Cancel routine for this IRP:
VOID
IrpqCancelRoutine(
IN PDEVICE_OBJECT DeviceObject,
IN PIRP Irp
)
{
IoReleaseCancelSpinLock( Irp->CancelIrql );

KeAcquireSpinLock( &pDevExt->IrpQ[q].SpinLock,
&kTempIrql );

RemoveEntryList( &Irp->Tail.Overlay.ListEntry );

KeReleaseSpinLock( &pDevExt->IrpQ[q].SpinLock,
kTempIrql );
}

In the Cancel routine, the call to RemoveEntryList might attempt to remove an IRP
that was previously removed by the dequeuing code. To correct this problem, the
Cancel routine must be able to detect whether the IRP is present in the queue. To

© 2003 Microsoft Corporation. All rights reserved.

 Cancel Logic in Windows Drivers - 24

make such detection possible, the driver could call InitializeListHead from the
dequeue code to initialize the link fields in the IRP; alternatively, it could mark the
IRP in some way. If the Cancel routine detects that the IRP has already been
removed, it must not try to remove the IRP from the queue.

Testing for Errors

The Windows DDK provides three driver development tools that can help you find
common IRP cancellation problems: the Driver Verifier, DC2, and Devctl.
The Driver Verifier checks for a wide variety of problems in drivers and provides
three levels of I/O verification, depending on the target operating system. Driver
Verifier can find many errors related to IRP cancellation, including the following:

 Calling IoCompleteRequest for an IRP while the Cancel routine is still set.
 Calling an IRP’s Cancel routine after the IRP has completed without
cancellation.
 Cancel routines that return at an elevated IRQL.
The DC2 and Devctl tools can assist in finding “lost” IRPs. A “lost” IRP is a request
that the device finished handling, but the driver neither completed by calling
IoCompleteRequest nor passed to another driver.

Call to Action and Resources

Call to Action for Driver Developers:
 In new drivers, use the cancel-safe IRP queuing routines (IoCsqXxx)
instead of traditional IRP-cancellation logic. These routines are documented in
the Windows DDK.
 In existing drivers, analyze cancel logic by tracing IRP ownership through
queuing and cancellation paths, and then correct race conditions if necessary.
 If possible, revise existing drivers to use the cancel-safe IRP queuing
(IoCsqXxx) routines.
 Use Driver Verifier, DC2, and Devctl to find other problems with Cancel
routines.
Resources:
 Windows Platform Development white papers and resources:
http://www.microsoft.com/hwdev/
 Windows Driver Development Kit:
http://www.microsoft.com/ddk
 Windows Logo Program for Hardware:
http://www.microsoft.com/winlogo/hardware/
 WHQL Test Specifications, HCTs, and testing notes:
http://www.microsoft.com/hwtest/

© 2003 Microsoft Corporation. All rights reserved.