CSE 870 Mini Project Concurrent Control with "Readers" and "Writers" P. Courtois, F. Heymans and D.

Parnas Communication of the ACM 14, 10 (Oct. 1971), 667668

Prashant Agrawal Vineet Bansal

List of Papers

Concurrent Control with "Readers" and "Writers" P. J. Courtois, F. Heymans and D. L. Parnas Communication of the ACM, 14, 10 (Oct. 1971), 667668. Concurrent Reading and Writing L. Lamport Communications of the ACM, 20,11 (Nov. 1977), 806811. Concurrent Reading While Writing G. Peterson ACM Trans. Programming Languages and Systems, 5, 1(Jan. 1983), 46-55. An Optimal Algorithm for Mutual Exclusion in Computer Networks G. Ricart, A. Agrawala Communications of the ACM, 24,1 (Jan. 1991), 9-17. A Three-Slot Asynchronous Reader/Writer Mechanism for Multiprocessor Real-Time Systems 1 Jing Chen and Alan Burns Tech. Rep. YCS-286, Department of Computer Science, University of York, Jan. 1997. The Non-Blocking Write Protocol NMW: A Solution to a Real-Time Synchronization Problem 2 G. Kopetz and J. Reisinger In Proceedings of the 14 th IEEE Symposium on Real-Time Systems, ( Dec. 1993), 131-137.

1 This paper does not cite [1] 2 This paper does not cite [1]

Concurrent Control with "Readers" and "Writers" P. J. Courtois, F. Heymans, and D. L. Parnas Objective A common paradigm in concurrent applications is isolation of shared data such as a variable, buffer, or document and the control of access to that data. The Readers/Writers problem is a typical example. This problem has two types of clients accessing the shared data. The first type, referred to as readers, only wants to read the shared data. The second type, referred to as writers, may want to modify the shared data. Exclusive write semantics is enforced, meaning that if a writer is active then no other writer or reader can be active. In this paper, Parnas et al. [1] propose two algorithms that allow concurrent reads while ensuring that writes are exclusive from each other and from reads. One solution attempts to have a minimal delay for reads, and the other to have a minimal delay for writes. Background Before describing the technique proposed in the paper, some background information about critical sections and semaphores is in order. A critical section is a part of a program where the shared memory is accessed. If we could arrange matters such that no two processes were ever in their critical sections at the same time, we could avoid race conditions i.e. conditions under which the final result of a computation depends on the scheduling of individual processes. A semaphore is a variable with two possible states. (Although semaphores with more states are possible, we will not discuss them here). The DOWN operation on a semaphore tests to check the value in the semaphore, and if it is found to be 1, sets it to 0. If the value is 0, the calling process blocks waiting for the value to become 1. The UP operation increments the value of the semaphore addressed. If one or more processes were sleeping on that semaphore, unable to complete an earlier DOWN operation, one of them is chosen by the system and is allowed to complete its DOWN operation. As it turns out, the notion of semaphores is invaluable in solving problems of mutual exclusion of processes. Technique The first solution is quite simple and ingenious: simply treat all readers as one process. The first reader who wants entry must obtain a main semaphore, seen by all processes. While at least one reader is in the critical section, other readers are allowed free entry and exit. The last reader to exit releases the semaphore and allows entry to a waiting writer. This solution has the fault that a writer may be blocked indefinitely while an infinite number of readers stream though the critical code. The second solution reverses this technique by allowing readers to block indefinitely while writers gain immediate access to the data. The first writer to capture access to the shared resource blocks other readers from entering the critical section. Application and Impact Algorithms to arbitrate concurrent accesses are at the heart of distributed operating systems. This seminal paper recognized for the first time the readers and writers problem that is an abstraction of concurrent controlled access to shared data. The algorithms presented here provided a starting point for researchers who have since tried to solve the problem in a more general setting. For example, Lamports algorithm [2] that always allows a writer to write without blocking, a technique useful in multiprocessor systems where mutual exclusion is an expensive solution, and Ricart and Agrawalas solution [4] that addresses overhead in a message passing system.

Concurrent Reading and Writing Leslie Lamport Lamport abstracts the essential aspects of algorithms dealing with sharing resources among a competing set of processes, by way of two general theorems. These theorems are of immense value in true concurrent environments where the problem of concurrent reading and writing of data cannot be accomplished by mutual exclusion alone, because serializing accesses may result in a considerable overhead. Thus the problem of concurrent reading and writing is considered here without introducing mutual exclusion. Theorems The basic notion in these theorems is that of versions of data. Let v denote a data item composed of one or more digits (a data unit whose reading and writing is atomic). Each operation that writes v begins with an initial version v[i] and ends with the final version, v[i+1]. We denote v = v1 vm to mean that v is composed of the data items vj. Each read (write) operation of v involves reading (writing) of each vj. Thus v[i] = v1[i]vm[i] If v consists of more than one digit, reading and writing v is not atomic: a read of v performed concurrently with a write to v may result in a value different from any of the versions v[i]. This value is said to contain traces of versions v[i1]v[im], in which case we denote its value by v[k,l] where k = minimum[i1,..,im] and l = maximum[i1,..,im]. The read operation obtains a consistent version v[k] of v if k=l. For lack of space we only present here the second and more practically applicable theorem put forward by Lamport: Theorem: Let v be an m-digit number (comprising of an m-integer sequence) and assume that i j implies v[i] v[j]. a. If v is always written from right to left, then a read from left to right obtains a value v[k,l] v[l]. b. If v is always written from left to right, then a read from right to left obtains a value v[k,l] v[l]. Although the proof of this theorem is involved, the basic idea implied here is, that writing data elements in one order and reading them in the opposite order allows us to solve the Readers/Writers Problem without assuming mutual exclusion of shared data. This theorem is insightful if concurrent reading is permitted on shared data, but a writer has exclusive access to the resource. This is a reasonable assumption since mutual exclusion of writers seems unavoidable. [2] The theorem helps us seek solutions in cases where the read operation is repeated in case when a reader obtains an incorrect result due to a concurrent write operation. Application to the Reader/Writer Problem The author proposes the solution to multiple readers/single writer situation, where read and write operations are allowed at any time. A reader checks the data to see if it might have obtained a faulty value due to a concurrent write operation. We maintain two versions of the data: v1 and v2. The writer increments v1 before it writes the data and it increments v2 after writing the data. The reader reads v2 before reading the data and v1 after reading it. In cases where these values are different, it attempts the read operation again. The critical point here is that the read and write operations on v1 or v2 be done in a mutually opposite order (right to left and left to right or vice versa). The proof is again non-trivial but the algorithm itself is simple and elegant. Such a technique might be desirable if we do not wish to keep the writer waiting for a reader to finish reading, and the probability of having to repeat a read operation is small, so that resorting to mutual exclusion is not economical.

Concurrent Reading While Writing Gary L. Peterson Abstract This paper discusses various algorithms that allow various processes to read shared data while another process is modifying that data. The algorithms are compared based on their time complexity and their space requirements. If more time efficient algorithms are used multiple copies of the shared data are required. With multiple copies of the shared data, the waiting time of the processes can be reduced. Six algorithms have been presented with various degrees of time efficiency and the corresponding minimum requirement for the number of copies of shared data is proved. The algorithms are centered towards distributed systems that try to extract maximum amount of parallelism from the system. Using the mutual exclusion techniques described in [1] effectively serializes the use of shared data thereby decreasing the parallelism that can be achieved. The solutions in this paper differ from [1] in that concurrent reading and writing is allowed on the shared data without resorting to mutual exclusion and thus the parallelism that can be achieved is increased. The principles presented in the paper are applicable to other models of concurrent programming such as message passing systems. Model and Techniques The solutions are applicable to problems where there are n readers and one writer that access shared data asynchronously. The shared data cannot be read or written atomically. To achieve this multiple copies of the shared data are stored in buffers. The readers should get the most recent copy of the shared data that has been written by the writer in entirety. If the data has not been completely written by the writer then it should not be visible to the reader. The first solution uses mutual exclusion to give exclusive right to the reader (writer) to read (write) the shared data. In this solution all the processes may have to wait but there are no lockouts. The solution allows more than one reader to read the data at the same time. The shared data does not have multiple copies. Only one copy is used by both the readers and the writer. The second solution makes the writer wait-free but the readers may be locked out. For example if the writer is continuously updating the shared data, the readers may never get a chance to read the data. This solution does not make multiple copies of the shared data either. The general principle used in this method is that the writer uses a flag to indicate that it is writing data. The reader checks the flag before and after reading the data and can conclude if the data was overwritten while it was reading the data. Provisions are also made to detect writes that start after the read starts and ends before the read ends. The next solution gives a method in which both the readers and writers are made wait-free at the cost of n+2 buffers. There are two sets of buffers: buff1 and buff2. The readers use buff1 to get a copy of the shared data. In buff2 each reader has a copy for itself that the writer uses to the store the value for the reader in case of concurrent reads and writes. The writer writes to buff1. It detects any overlapping reads and makes copies for those readers and places them in buff2. The reader reads the value from buff1 and then from buff2. It checks if buff2 is a copy of buff1 and if so it uses that value. If buff2 and buff1 differ then it uses shared variable to determine if buff1 is correct and then uses its value. A theorem stating that a minimum of n+2 buffers are required for a Concurrent Reading while Writing (CRW) problem which requires wait-free readers and writers is proved. The drawback of this method is that every time a writer writes to the shared memory it might be required to update n copies of the shared data. A workaround this problem is also suggested using binary flags and garbage collectors. The other three solutions show the trade-off between the waiting times and the number of buffers if they are reduced to fewer than n+2. The writer is always wait-free but the readers are made to wait either on other readers or the writer. The fourth solution requires n+1 buffers. The readers have to wait on the writer but they do not wait on other readers. The fifth solution makes the readers wait for other readers but not for the writer. This solution uses only three buffers. The last solution uses only 2 buffers but now the readers are made to wait for any process. Conclusion The various solutions presented for CRW require buffers proportional to the number of readers if a small amount of waiting time is permitted. If the waiting time permitted is large then the numbers of buffers required can be reduced considerably. For any wait-free solution for both readers and writers a minimum of n+2 buffers are required.

An Optimal Algorithm for Mutual Exclusion in Computer Networks Glenn Ricart, Ashok Agrawala Ricart and Agrawala [4] propose an algorithm to solve the mutual exclusion problem in a distributed system. The individual nodes of the distributed network communicate by sending messages, with no shared memory. The algorithm is historic because of the authors claim that no algorithm uses fewer messages, operates faster, and exhibits concurrent, symmetric and distributed control. [4]. The proposed algorithm assumes total ordering of messages in the distributed system, which is easily achieved by using a combination of node number and sequence number of requests. Each node has its own unique node number. Technique A node attempting to invoke mutual exclusion builds a REQUEST message containing the name of the critical section it wishes to enter, its node number and a sequence number of the request. The sequence number of a request is generated by the requesting node, which initializes this number to be one greater than the highest numbered sequence number in any REQUEST message it has received. When a node receives a REQUEST message from another, it has three courses of action: 1. 2. 3. If the receiver is not currently holding the critical section and does not wish to enter it, a REPLY message is immediately sent to the sender. If the receiver has already entered the critical section, it defers a REPLY. Instead, it queues the request. If the receiver has not yet entered the critical section but wishes to do so, a priority order decision is made by comparing the sequence number of the incoming request with the sequence number of the REQUEST message it has sent out. The lower numbered request wins. In case of a tie, the node numbers are compared to determine which will enter first. The lower numbered node wins.

After sending out requests to enter a critical section, a node waits for every other node to send a REPLY message. As soon as all replies are in, it may enter the critical section. On leaving the critical section, it sends a REPLY message to all waiting nodes in its queue and deletes them all from the queue. The sequence numbers and node numbers thus form an ordering among requesting nodes. The authors empirically prove that deadlock and starvation is impossible in such a total ordering of requests. Furthermore, for a network of N nodes, the algorithm uses 2(N-1) messages for each critical section invocation, which also turns out to be the minimum required for a network with independent and concurrent nodes. The Readers and Writers problem is solved in the case where writers are given priority, by a simple modification: readers never defer a REPLY for another reader but always reply immediately. Writers follow the original algorithm. The authors propose several enhancements to the algorithm. Although sequence numbers ensure that highernumbered nodes are not starved by low-numbered nodes, a modification to the algorithm where sequence numbers are incremented by a random integer instead of the fixed increment of 1, ensures that lower numbered nodes are not favored in the decision making process. Another effective optimization in sparse networks is achieved by imposing an upper bound on the message transmission time between nodes. This means that an implicit REPLY is assumed if no REPLY is obtained within a fixed time frame. In such a scheme. an explicit message called DEFFERRED is sent when REPLY would ordinarily not be sent. The number of messages in this implicit reply scheme varies from (N-1) to 3(N-1), and the scheme works well when there is little contention for resources.

A Three-Slot Asynchronous Reader/Writer Mechanism for Multiprocessor Real-Time Systems Jing Chen and Alan Burns Abstract This paper extends the concept of a general Reader/Writer problem to Single Reader/Single Writer problem for a real-time system. In real-time systems the response time of the system to various events is critical. It is highly desirable in a real-time system that the processes are not blocked. The mechanism presented in the paper is fully asynchronous and allows the reader and the writer to access the shared data in a loop-free and wait-free manner. The mechanism uses an atomic "read-modify-store" operation. More specifically it assumes a compare-and-swap instruction in hardware. The mechanism does not impact the timing behavior of tasks and it also reduces the blocking problems no process can be locked out indefinitely, and the priority inversion problem: a problem in which a higher priority process waits for a process of lesser priority, is eliminated. Model and Technique The traditional solution to the Readers/Writers problem has been to use a locking mechanism where other processes are blocked until the process using the shared space releases the lock. This introduces waiting delay that degrades the performance of a real time system. Another solution is to allow asynchronous use of shared memory and use read-and-check method where after every read - the reader checks if any writer was writing data concurrently. This introduces loop delay. The solution assumes a multiprocessor real time system that uses shared memory. A number of time bound tasks run on the multiprocessor system. The execution of each task should be completed before its deadline. The integrity of the data read by the reader should be maintained and the reader should get the latest version of the data. Also no assumptions are made about the speed and duration of the reads and writes. There should at least be one instruction that allows atomic read and write operations on the shared memory. The mechanism requires 3 buffers slots. This allows the writer to choose from two buffers when it wants to write and leaves 1 slot from which the reader can read. There are 2 control variables. One control variable (LATEST) is used to indicate the buffer slot that has the latest version of the data. The second control variable (READING) is used to indicate the buffer slot that the reader is reading from. The READING variable can be updated by either the writer or the reader. The READING variable is also used to indicate the state of the reader. A zero value in the control variable of the reader indicates that it is preparing to read the data. Any non-zero value indicates that the reader is reading the data or is inactive. The reader updates the control variable to zero to indicate it is going to read the data and then it changes the control variable to a non-zero value to indicate the buffer it is reading from (which is the value of LATEST). If a writer after completing the update of the shared data discovers that a control variable for a reader is zero, it can conclude that the reader is about to read the data but has not started to do so. It can replace the contents of this control variable with the position of the buffer having the latest copy of the data. It does this using the atomic read-and-write instruction ("compare-and-swap" operation). Before the reader moves the number of the buffer it is reading to its control variable, it checks to see if any writer has updated its control variable. If so then it uses the buffer number written by the writer. Thus the reader always gets the latest version on the data. Conclusion The primary consideration for any solution for simultaneous access to shared data in a real time system is to make the processes lock-free and wait-free. Mutual exclusion solutions serialize the access and decrease the parallelism. Read-and-check methods increase the execution time of the readers. A mechanism is described which achieves this objective at the cost of 3 copies of the data that is the lower bound for a wait-free single reader/writer problem. The writer writes to only one buffer and reader only reads from one buffer.

The Non-Blocking Write Protocol NBW: A Solution to a Real-Time Synchronization Problem H. Kopetz and J. Reisinger Abstract This paper presents two solutions for the problem of multiple readers and a single writer for a hard-real time system. The protocols are non-blocking: neither the writer nor the readers have to be blocked if they want to access the shared data. A schedulability analysis of the task set (readers and writers) using these protocols is also given. Using the protocols described in this paper, the tasks do not block and this reduces the complexity of the analysis. Both the protocols in this paper use read-and-check method for the readers. The readers may have to loop multiple times to read the shared data if the writer is continuously updating the contents of the shared memory before a reader completes its read operation. The first method does not use buffers for multiple copies of the shared data. The second protocol tries to overcome the drawbacks of first method by making use of multiple buffers for the data. The protocol can be adapted to a fixed number of buffers as per application requirements. This involves a tradeoff between the excess memory requirements and execution time. Model and Technique The methods assume a distributed real-time system made of multiple node. There is also a dual-ported memory that is shared by all nodes.. All the read and write messages go to a controller that operates on the dual-ported memory. The techniques described in the paper can also be used for a variable in a programming language environment. There are a set of tasks for each CPU that have a deadline and a maximum execution time. The general properties of reader/writer problems are also assumed, that is a reader should always get a valid copy of the shared data. Also all the tasks have to be completed before their deadlines and a reader cannot block a writer. The information flow from the writer to the reader is unidirectional. No changes need to be made to the writer if any reader is added or removed from the system. In the first solution the writer can write any time it wants to write. A reader cannot invalidate the shared data. The reader on the other hand must check at the end of the read operation whether any write operation overwrote the data while it was reading. In this case the reader has to restart the operation. There is a control field CCF that is initialized to zero. A writer, before it begins to write the data, increments the CCF by 1. After it completes writing the data, the CCF is again incremented by 1. There is an upper bound for maximum value of CCF after which the value wraps around. The reader on the other hand reads the value of CCF before it begins to read the data. Then it reads the data and again reads the value of CCF. If the two values differ or if the first value is odd, it concludes that an overlapping write took place and it restarts the process. The first protocol has a drawback of not being able to handle tasks that do have any laxity for read-retries. Laxity is the time difference between the maximum amount of execution time required by the task and its deadline. Also if a read call takes an appreciable portion of execution time then executing read multiple times makes this protocol inefficient. To overcome this, a second protocol is suggested that requires buffers to store multiple copies of the data. In this protocol the readers and the writer uses the CCF to indicate the number of the buffer that is being accessed presently. There is also another control variable bcnt that is used to indicate the number of buffers reserved for the data. The writer accesses the buffers in cyclic manner. It increments CCF by 2, but it uses (CCF/2 mod bcnt) as the number of buffer to store the data. Thus CCF/2 mod bcnt always specifies the buffer that is presently been written or will be written and hence (CCF/2 1) mod bcnt gives the number of the buffer having the latest version of the data. An instance of message is guaranteed not to be overwritten if less than bnct writes have taken place. Thus the difference between the two CCF values that the reader reads should be less than bcnt*2 - 2 (every write increments CCF by 2 and there should be maximum of bcnt 1 writes). Conclusion Two non-blocking algorithms are presented for multiple reader/single writer problems for a hard-real time system. Both the solutions could require multiple read-retries. But using more memory in the second solution can reduce the number of retries.

References 1. 2. 3. 4. 5. P. Courtois and D. Parnas. Concurrent Control with "Readers" and "Writers". Communication of the ACM 14, 10 (Oct. 1971), 667668 L. Lamport. Concurrent reading and writing. Communications of the ACM, 20(11):806--811, November 1977. G.L. Peterson. Concurrent reading while writing. ACM Trans. Programming Languages and Systems, 5: 1(1983), 46-55. G. Ricart and A. Agrawala. An optimal algorithm for mutual exclusion in computer networks. Communications of the ACM, 24(1):9-17, 1981. Chen, J., and Burns, A. A Three-Slot Asynchronous Reader/Writer Mechanism for Multiprocessor Real-Time Systems. Tech. Rep. YCS-286, Department of Computer Science, University of York, Jan. 1997. H. Kopetz and J. Reisinger. The non-blocking write protocol NBW: A solution to a real-time synchronization problem. In Proceedings of the 14 th IEEE Symposium on Real-Time Systems, pp. 131-137. December 1993.

6.