We present an efﬁcient and practical lock-free imple-mentation of a concurrent priority queue that is suitable for both fully concurrent (large multi-processor) systemsas well as pre-emptive (multi-process) systems. Many al-gorithms for concurrent priority queues are based on mu-tual exclusion. However, mutual exclusion causes block-ing which has several drawbacks and degrades the system’soverallperformance. Non-blockingalgorithmsavoidblock-ing, and are either lock-free or wait-free. Previously knownnon-blocking algorithms of priority queues did not performwell in practice because of their complexity, and they areoftenbased onnon-availableatomic synchronizationprimi-tives. Our algorithm is based on the randomized sequentiallist structure called Skiplist, and a real-time extension of our algorithm is also described. In our performance evalu-ation we compare our algorithm with some of the most efﬁ-cient implementations of priority queues known. The exper-imental results clearly show that our lock-free implemen-tation outperforms the other lock-based implementations inall cases for 3 threads and more, both on fully concurrent as well as on pre-emptive systems.
Priority queues are fundamental data structures. Fromthe operating system level to the user applicationlevel, theyare frequently used as basic components. For example, theready-queue that is used in the scheduling of tasks in manyreal-time systems, can usually be implemented using a con-currentpriority queue. Consequently,the design of efﬁcientimplementations of priority queues is a research area thathas been extensively researched. A priority queue supportstwo operations, the
operation.The abstract deﬁnition of a priority queue is a set of key-value pairs, where the key represents a priority. The
operation inserts a new key-value pair into the set, and the
operation removes and returns the value of thekey-valuepair with the lowest key(i.e. highest priority)thatwas in the set.Toensureconsistencyofashareddataobjectinaconcur-rent environment, the most common method is to use mu-tual exclusion, i.e. some form of locking. Mutual exclusiondegrades the system’s overall performance  as it causesblocking, i.e. other concurrent operations can not make anyprogress while the access to the shared resource is blockedby the lock. Using mutual exclusion can also cause dead-locks, priority inversion (which can be solved efﬁciently onuni-processors  with the cost of more difﬁcult analysis,although not as efﬁcient on multiprocessor systems )and even starvation.To address these problems, researchers have proposednon-blocking algorithms for shared data objects. Non-blocking methods do not involve mutual exclusion, andtherefore do not suffer from the problems that blockingcan cause. Non-blocking algorithms are either lock-freeor wait-free. Lock-free implementations guarantee that re-gardless of the contention caused by concurrent operationsand the interleaving of their sub-operations, always at leastone operation will progress. However, there is a risk forstarvation as the progress of other operations could causeone speciﬁc operation to never ﬁnish. This is although dif-ferent from the type of starvation that could be caused byblocking, where a single operation could block every otheroperation forever,and cause starvation of the whole system.Wait-free  algorithms are lock-free and moreover theyavoid starvation as well, in a wait-free algorithm every op-eration is guaranteed to ﬁnish in a limited number of steps,regardless of the actions of the concurrent operations. Non-blocking algorithms have been shown to be of big practicalimportance in practical applications [17, 18], and recentlyNOBLE, which is a non-blocking inter-process communi-cation library, has been introduced .There exist several algorithms and implementations of concurrent priority queues. The majority of the algorithmsare lock-based, either with a single lock on top of a se-quential algorithm, or specially constructed algorithms us-ing multiple locks, where each lock protects a small part of the shared data structure. Several different representationsof the shared data structure are used, for example: Hunt
 presents an implementation which is based onheap structures, Grammatikakis
 compares differ-ent structures including cyclic arrays and heaps, and mostrecently Lotan and Shavit  presented an implementa-tion based on the Skiplist structure . The algorithmby Hunt
locks each node separately and uses a tech-nique to scatter the accesses to the heap, thus reducing thecontention. Its implementation is publicly available andits performance has been documented on multi-processorsystems. Lotan and Shavit extend the functionality of theconcurrent priority queue and assume the availability of aglobal high-accuracy clock. They apply a lock on eachpointer, and as the multi-pointer based Skiplist structureis used, the number of locks is signiﬁcantly more than thenumber of nodes. Its performancehas previously only beendocumented by simulation, with very promising results.Israeli and Rappoport have presented a wait-free algo-rithm for a concurrent priority queue . This algorithmmakes use of strong atomic synchronization primitives thathave not been implemented in any currently existing plat-form. However, there exists an attempt for a wait-free al-gorithm by Barnes  that uses existing atomic primitives,though this algorithm does not comply with the generallyaccepted deﬁnition of the wait-free property. The algo-