Chapter 3 - Synchronizing Pthreads

Pthreads Programming Bradford Nichols, Dick Buttlar and Jacqueline Proulx Farrell Copyright ╘ 1996 O'Reilly & Associates, Inc.

Synchronization in the ATM Server
To wrap up our discussion of mutex and condition variables, we'll return to our ATM example. In our discussion of mutexes earlier in this chapter, we added a single mutex to the example to protect the bank account database. As we noted at the time, this isn't the best way to impose synchronization inasmuch as it allows only one thread to access the database at a time. In this section, we'll provide a more optimal solution to our ATM server's synchronization problems. We'll focus on the following three areas: ∙ Synchronizing access to the bank account database ∙ Limiting the number of concurrent worker threads ∙ Controlling the shutdown of the server We'll continue to use mutex variables to synchronize access to account data. Imposing a limit on the number of simultaneously active worker threads and controlling server shutdown are event-driven tasks; we'll use both mutexes and condition variables when implementing them. We first encountered the ATM server example in Chapter 2. We designed it according to the classic boss/worker model for a multithreaded program. In our server, the boss creates a new thread for each request it receives (be it a deposit, withdrawal, or balance inquiry), and the worker thread processes the request independently of the boss or any other worker thread. We've done only half the job by creating threads and adding concurrency to the server. Now we'll finish up by adding robust and efficient synchronization mechanisms.

Synchronizing Access to Account Data
Our multithreaded ATM server must contend with many potential race conditions between worker threads accessing account data. We expect its deposit and withdraw operations to be atomic. Because of this, in Example 3-1 we added a single mutex to the server to protect the integrity of the accounts database. Although simple, this approach has major performance limitations. Because every worker thread accesses the database and only one at a time can lock the mutex, only one thread can be executing each time an account balance changes. When the server is heavily loaded, the new result is that it behaves very like a single-threaded program. Because different requests frequently access different accounts in the database, multiple requests could often execute at the same time without interfering with each other. In this light, the single-mutex approach is overly conservative. A much more appropriate solution would be to use finer-grained locking on our data. Thus, we'll associate a mutex with each database account. The cleanest way to proceed with this decision would be to redesign the ATM server's database module to include mutexes in the account structures themselves. For our purposes, let's assume that the database module is legacy code and we can't≈or don't want to≈modify it. Instead, we'll place the mutex in a separate structure outside the database module. In the code fragment in Example 3-15, we'll implement our locking scheme. We'll globally define an array of mutex variables, called account_mutex, that has an element for each account. Because accounts have IDs between zero and MAX_NUM_ACCOUNTS, we'll use the account ID as an index into the mutex array. The server's main routine will initialize the mutex array by calling atm_server_init. Example 3-15: Initializing per-account locks for the ATM database (atm_svr.c) pthread_mutex_t account_mutex[MAX_NUM_ACCOUNTS]; . . . void atm_server_init(int argc, char **argv) { . . . for (i = 0; i < MAX_NUM_ACCOUNTS; i++) pthread_mutex_init(&account_mutex[i], NULL); . . . }

converted by

account_t *accountp. /* Parse input string */ sscanf(req_buf. "%d %s". "%d %d %d %s". . int rtn. &amount). . atm_err_tbl[-rtn]). } pthread_mutex_lock(&account_mutex[id]). sets the flag. These systems must limit the number of threads each user may run at a time. The new_account routine looks for the first account whose in-use flag is clear. There is symmetry in creating an object and destroying an object: both require the same kind of protection. . /* Get a new account */ if ((rtn = new_account(&id. .c) void deposit(char *req_buf. . There are some good reasons for doing so. a worker thread need lock only the mutex for the specific account it is accessing. pthread_mutex_lock(&create_account_mutex). as shown in Example 316. /* Check inputs */ if ((id < 0) || (id >= MAX_NUM_ACCOUNTS)) { sprintf(resp_buf. you reach a practical limit at the point you find that you're getting diminishing returns by creating more and more threads. */ } pthread_mutex_unlock(&account_mutex[id]). . &temp. id. Which mutex should it lock? The account doesn't exist yet. /* Code to update and access account balance. The number of worker threads is determined and fixed at initialization. "%d %s". converted by Web2PDFConvert. with this set of mutexes. . other threads can concurrently lock other mutexes and access other accounts. just as it manages processes. /* Retrieve account from database */ if ((rtn = retrieve_account( id. Both could return with the same account ID for two different customer accounts≈not a good idea.* We'll examine this phenomenon further in our performance measurements in Chapter 6. * Thread pools don't have this problem. The database contains a list of potential accounts. password. . Even if your system imposes no limit or an extremely high one. On some operating systems. Limiting the Number of Worker Threads Our next synchronization task will be to limit the number of worker threads that can exist at a single time. atm_err_tbl[-rtn]). To remove this hazard. TRANS_FAILURE. &id. Here is fertile ground for a classic race condition. Any thread wishing to add an account must hold this mutex (which we've globally defined) before proceeding. The revision of the create_account routine in Example 3-17 shows the new mutex. amount. Example 3-17: A Special Mutex for Opening New Accounts (atm_svr. and plugs in the information about the new account. &password. int temp. &accountp)) < 0) { sprintf(resp_buf. we'll need an additional mutex. If two threads execute new_account concurrently. the kernel manages threads as separate contenders for the CPU. the worker threads are created. they could interleave their flag-reading and flag-setting. each with a flag indicating whether or not it's in use. Example 3-16: Using Per-Account Locks for the ATM Database (atm_svr. char *resp_buf) { int rtn. . } The thread that runs our create_open routine to create a new account poses a special problem. and the worker thread has no account ID to use! Let's look at how the database layer of our ATM server actually creates a new account. &accountp)) < 0) { sprintf(resp_buf. -1.c) pthread_mutex_t create_account_mutex = PTHREAD_MUTEX_INITIALIZER. . -1. } Note that deleting an account will work the same way. ERR_MSG_BAD_ACCOUNT).Now. and they live for the duration of the program. account_t *accountp. } pthread_mutex_unlock(&create_account_mutex). TRANS_FAILURE. TRANS_FAILURE. At this point. "%d %d %d %d ". void create_account(char *resp_buf) { int id. It no longer needs to lock up the entire database.

thread_info_t worker_info. converted by Web2PDFConvert. .) { /*** Wait for a request ***/ workorderp = (workorder_t *)malloc(sizeof(workorder_t)). pthread_mutex_t mutex.c) #define MAX_NUM_THREADS 10 . . "%d". the boss increments the counter. unlocks the mutex. int trans_id. unlocks the mutex. return 0.To limit the number of worker threads.mutex). pthread_cond_t thread_exit_cv.num_active == MAX_NUM_THREADS) { pthread_cond_wait(&worker_info. extern int main(argc. . The server's main routine will set the counter to zero and initialize the mutex and the condition variable. . we'll need to keep a count of them.c) void process_request(workorder_t *workorderp) { char resp_buf[COMM_BUF_SIZE]. { workorder_t *workorderp.mutex).. resp_buf). pthread_t *worker_threadp. Example 3-19: Limiting the Number of Worker Threads≈Workers (atm_svr. pthread_mutex_unlock(&worker_info. &worker_info. The boss thread increments it when it creates a new worker. . we'll modify our ATM to add a worker_info structure. . . when the boss thread receives a request. If the limit has been reached. We'll synchronize access to the counter using a mutex. sscanf(workorderp->req_buf. typedef struct { int num_active. a mutex (num_active_mutex). the boss waits on the thread_exit_cv condition variable.. /*** Have we exceeded our limit of active threads ? ***/ pthread_mutex_lock(&worker_info. switch(trans_id) { case OPEN_ACCT_TRANS: open_account(resp_buf). it locks the worker_info mutex and checks the count of active workers before creating a new worker thread. workorderp>req_buf). and continues. int conn. break. and each worker decrements it when it exits. . It'll include a counter (num_active). char **argv. pthread_mutex_lock(&worker_info. for(. &trans_id). . and a condition variable (thread_exit_cv). Both boss and worker threads must access this counter.. } worker_info.mutex). Example 3-18: Limiting the Number of Worker Threads≈Boss (atm_svr. . argv). free(workorderp). } server_comm_shutdown(). . } server_comm_send_response(workorderp->conn. In Example 3-19.mutex). When the condition is signaled the boss wakes up and rechecks the counter. If the count of active workers is now below the limit.thread_exit_cv. while (worker_info. the boss increments the counter. If the number of active workers has not yet reached its limit. .com . }thread_info_t. atm_server_init(argc. . we'll adjust the process_request code our worker threads execute. argv) int argc. int trans_id. } Now. . server_comm_get_request(&workorderp->conn. /*** Spawn a thread to process this request ***/ pthread_create(worker_threadp.num_active++. In Example 3-18. and continues.

converted by Web2PDFConvert. . argv) int argc. If not.thread_exit_cv). the boss will request the signal when the active worker count reaches zero.) { /*** Wait for a request ***/ workorderp = (workorder_t *)malloc(sizeof(workorder_t)). server_comm_get_request(&workorderp->conn. When it's awakened. If the final worker has exited. (Of course. . at some time. .) { . &worker_info. the boss will stop creating new threads so that this can eventually happen.) We'll modify the main routine in the boss thread. it calls pthread_cond_signal to signal thethread_exit_cv condition variable to the waiting boss thread.c) process_request(. int trans_id. If the counter is greater than zero. /* process it here with main() thread */ if (shutdown_req(workorderp->req_buf. } } /*** Have we exceeded our limit of active threads ? ***/ pthread_mutex_lock(&worker_info. we'll reuse the active worker counter and the thread_exit_cv condition variable. If the active worker counter is zero. } Each worker thread must decrement the active worker count when it exits. the boss unlocks the mutex.mutex).num_active > 0) { pthread_cond_wait(&worker_info. the system terminates the process and all its threads.num_active == (MAX_NUM_THREADS .c) extern int main(argc.1)) pthread_cond_signal(&worker_info. If it finds that it has decremented the counter to one less than the limit.mutex). for(. and the boss proceeds to shut down the program. &trans_id). } When the boss thread receives a shutdown request.mutex).worker_info.thread_exit_cv. We can't allow this to happen. . as shown in Example 3-20. To make sure that all worker threads get to complete active tasks before the boss thread exits main. thus terminating the program. if (worker_info. return 0. . This time. argv). if (trans_id == SHUTDOWN) char resp_buf[COMM_BUF_SIZE].. the boss must wait for the thread_exit_cv condition variable to be signaled.mutex). } server_comm_shutdown(). int conn. the count is zero. the boss thread requested a signal when the active worker count was one less than the active worker limit. char **argv. as well as when it decrements the worker count to one below the limit. resp_buf)) { server_comm_send_response(workorderp->conn. the boss must wait on the condition variable again. so our final synchronization task will be to handle server shutdown more gracefully. "%d". resp_buf). /* Wait for in-progress requests threads to finish */ while (worker_info. pthread_t *worker_threadp. When the boss thread finishes main.num_active--. It does this in the process_request routine. pthread_mutex_lock(&worker_info. workorderp>req_buf). the boss thread runs our program's main routine. Synchronizing a Server Shutdown In the current version of our ATM server. including those worker threads that are still processing active requests.mutex). the boss rechecks the active worker count. { workorder_t *workorderp. } pthread_mutex_unlock(&worker_info. atm_server_init(argc. and leaves the main loop.. free(workorderp). We'll modify our process_request routine in Example 3-21 so that each worker thread signals the thread_exit_cv condition variable before it exits. Example 3-20: Processing a Shutdown in the Boss Thread (atm_svr. When we used them to control the number of concurrent workers. Example 3-21: Processing a Shutdown in the Worker Thread (atm_svr. pthread_mutex_unlock(&worker_info. /*** Is it a shutdown request? ***/ sscanf(workorderp->req_buf. runs a cleanup function. it locks the worker_info mutex and checks the active worker counter..

free(workorderp). resp_buf). we could define a new condition variable to indicate when the active worker count reaches zero. Instead of using the thread_exit_cv condition variable for shutdown handling.mutex)..thread_exit_cv). server_comm_send_response(workorderp->conn. pthread_cond_signal(&worker_info. If the boss thread is waiting on the condition. Inc ╘ 2000 √ Feedback converted by Web2PDFConvert. .mutex). pthread_mutex_unlock(&worker_info. Although the boss can proceed with program shutdown only when the last worker has exited. each exiting worker thread will wake it up (and it will go right back to sleep) until the last worker decrements the active worker counter to 0.num_active--. the boss will wake up and reenter its wait nine times before it can finally do something useful! We could fix this. pthread_mutex_lock(&worker_info. each worker would call pthread_cond_signal on our new condition variable if it notices that the count has become it will wake up and shut down the program. Books24x7. As it exits. If ten worker threads are active when their boss receives the shutdown request. . } This works fine but is a bit inefficient.

Sign up to vote on this title
UsefulNot useful