Interprocess Communication &

Process Synchronization
Fall 09

1
Cooperating Processes
• Independent process cannot affect or be affected by the
execution of another process
• Cooperating process can affect or be affected by the
execution of another process
• Advantages of process cooperation
– Information sharing
– Computation speed-up
– Modularity
– Convenience
– Cooperating processes need interprocess
communication (IPC) and process synchronization
2
Communications Models

Message passing Shared memory

3
Producer-Consumer Problem
 A paradigm for cooperating processes
 Producer process produces information that is consumed by a
consumer process
 E.g. web server and browser

 If shared memory is available

 Producer write to the buffer, client read from the buffer
 unbounded-buffer places no practical limit on the size of the
buffer
 bounded-buffer assumes that there is a fixed buffer size

4
 Bounded-Buffer – Shared-Memory Solution

 Shared data: implemented as a circular array

#define BUFFER_SIZE 10
typedef struct {
. . . // information to be shared
} item;
out in
0
item buffer[BUFFER_SIZE];
int in = 0;
int out = 0;

5
 Bounded-Buffer
Producer out
in
while (true) {
/* Produce an item */
while (( (in + 1) % BUFFER_SIZE) == out)
; /* do nothing -- no free buffers */
buffer[in] = newProducedItem;
in = (in + 1) % BUFFER SIZE;
}

while (true) {
while (in == out)
; // do nothing -- nothing to consume
Consumer
// remove an item from the buffer
itemToConsume = buffer[out];
out = (out + 1) % BUFFER SIZE;
return itemToComsume;
}
Solution is correct, but can only use
6
BUFFER_SIZE-1 elements
 POSIX Shared Memory
– Process first creates shared memory segment
segment_id= shmget(IPC_PRIVATE, size, S_IRUSR | S_ IWUSR);

shmget - allocates a shared memory segment SYNOPSIS

#include <sys/ipc.h>
#include <sys/shm.h>
Int shmget(key_t key, int size, int shmflg)
shmget() returns the identifier of the shared memory segment associated to
the value of the argument key. A new shared memory segment, with size
equal to the round up of size to a multiple of PAGE_SIZE, is created if key
has value IPC_PRIVATE or key isn't IPC_PRIVATE, no shared memory
segment is associated to key, and IPC_CREAT is asserted in shmflg (i.e.
shmflg&IPC_CREAT isn't zero).

7
POSIX Shared Memory
– Process wanting access to that shared memory must attach
to it
shared_memory = (char *) shmat(id, NULL, 0);
SYNOPSIS
#include <sys/types.h>
#include <sys/shm.h>
void * shmat(int shmid, const void *shmaddr, int shmflg);
The function shmat attaches the shared memory segment identified by shmid
to the address space of the calling process. The attaching address is
specified by shmaddr with one of the following criteria:
 if shmaddr is NULL, system chooses a suitable (unused) address at which to
attach the segment
 if shmaddr isn’t NULL, ….

8
POSIX Shared Memory
- Now the process could write to the shared memory
sprintf(shared_memory, "Writing to shared memory");

– When done a process must detach shared memory from its

address space
shmdt(shared memory);
shmdt detaches the shared memory segment located at the address
specified by shmaddr from the address space of the calling process. The to-
be-detached segment must be currently attached with shmaddr equal to the
value returned by the its attaching shmat call.

9
Sample code studies
 Handout for share memory examples
 Two unrelated processes communicate through shared memory
 Access shared memory is simpler between related
processes, i.e., parent and child
 child process is a copy of parent process, shared-memory
attached to parent is also attached to child’s address space
Case studies: IPC with shared memory
 Calculating and displaying Fibonacci sequence(P.145,
3.19)
 Use command line argument to specify how many elements to
calculate & output
 Create a child process to calculate the sequence elements
 Parent process displays the result to standard output
Producer/Consumer as processes
 They have their different address space
 Use shared memory to implement the circular buffer
struct circular_buffer{
item buffer[BUFFER_SIZE];
int in;
int out;
};
circular_buffer * shared_memory;

segment_id = shmget(key, sizeof(circular_buffer),S_IRUSER|S_IWUSER);

shared_memory = (circular_buffer *)shmat (segment_id, NULL, 0);
…
Message Passing

• Message system – processes communicate with each other

without resorting to shared variables
• Mechanism for processes to communicate and to synchronize
their actions
• If proceses P and Q wish to communicate, they need to:
– establish a communication link between them
– exchange messages via send/receive
– send(message) – message size fixed or variable
– receive(message)
– Implementation of communication link
– physical link: how is message passing link is implemented ? e.g.,
shared memory, hardware bus, network connection …
– logical link: focus on logical properties of the link

13
Communication link: Design Questions
• How are links established?
• Direct Communication: process P send message to Q
• Indirect Communication: through a mailbox, message queue, …
• Can a link be associated with more than two processes?
• How many links can there be between every pair of
communicating processes?
• What is the capacity of a link?
• Is the size of a message that the link can accommodate
fixed or variable?
• Is a link unidirectional or bi-directional?

14
Direct Communication
 Processes must name each other explicitly:
 send (P, message) – send a message to process P
 receive(Q, message) – receive a message from process
Q
 Properties of direct communication
 Links are established automatically
 A link is associated with exactly one pair of
communicating processes
 Between each pair there exists exactly one link
 Link may be unidirectional, but is usually bi-directional

15
Indirect Communication
 Messages are directed and received from
mailboxes (also referred to as ports), or queues
 Each mailbox has a unique id
 Processes can communicate only if they share a
mailbox
 Properties of communication link
 Link established only if processes share a common
mailbox
 A link may be associated with many processes
 Each pair of processes may share several
communication links
 Link may be unidirectional or bi-directional

16
Indirect Communication
 Operations
 create a new mailbox
 send and receive messages through mailbox
 destroy a mailbox
 Primitives are defined as:
send(A, message) – send a message to
mailbox A
receive(A, message) – receive a message
from mailbox A

17
Indirect Communication
• Mailbox sharing
– P1, P2, and P3 share mailbox A
– P1, sends; P2 and P3 receive
– Who gets the message, P2, P3 or both?
• Solutions
– Allow a link to be associated with at most two processes
– Allow only one process at a time to execute a receive operation
– Allow the system to select arbitrarily the receiver. Sender is
notified who the receiver was.

18
Synchronization
 Message passing may be either blocking or non-
blocking
 Blocking is considered synchronous
 Blocking send has the sender block until message is
received
 Blocking receive has the receiver block until a
message is available
 Blocking message passing is a way to synchronize two
processes
 Non-blocking is considered asynchronous
 Non-blocking send has the sender send message and
continue
 Non-blocking receive return immediately, with a valid
message or null (if no message is available)

19
Asynchrony (Synchrony): from Wikipedia
 The state of not being synchronized
 Digital logic and physical layer of communication
 asynchronous process does not require a clock signal
 Data link layer of communication
 asynchrony is synonym of statistical multiplexing: information
transmission may or may not start immediately as requested by
sender, additional delay being caused by medium congestion
 Programming
 asynchronous events are those occurring independently of main
program flow
 E.g. Unix signal is asynchronous event to a process
 asynchronous actions are actions executed in a non-blocking
scheme, allowing main program flow to continue processing
 Asynchronous I/O: Interrupt I/O
 Buffering
 Queue of messages attached to the link; implemented in
one of three ways
1. Zero capacity – 0 messages
Sender must wait for receiver (rendezvous)
2. Bounded capacity – finite length of n messages
Sender must wait if link full
3. Unbounded capacity – infinite length
Sender never waits

21
Code studies: POSIX Message Queue
 POSIX messaging:
 Two or more processes can exchange information via access to
a common system message queue.
 Processes must share a common key to access the queue
 Initialize queue: msgget()
 Send/receive message: msgsnd, msgrcv() functions, can be
blocking or non-blocking
 Handout
Pipes
 first-in-first-out queue, written by one process and read by
another.
 Design choices
 Unidirectional or bidirectional communication?
 If bidirectional, is it half duplex (data travel in one way at a
time) or full duplex?
 Must the processes be related to use the pipe?
 Must the processes reside in same machine ?
Unix Unnamed Pipe
 Unidirectional: info. flow from one end to another end
 Allow parent-child processes to communicate with each other
 int pipe(int fd[2]); // a system call
 you pass it an array of two integers.
 on success, zero is returned, fd contains two file descriptors,
fd[0], fd[1]
 On error, -1 is returned, and errno is set appropriately.
 Routine perror() produces a message on standard error
output, describing last error encountered during a system
call or library function.

fd[0]: reading end fd[1]: writing end

Unix Unnamed Pipe
int fd[2];
if (pipe(fd)!=0){
fd[0]: reading end fd[1]: writing end
perror (“pipe”);
exit 1;
} else {
// on success, fd contains two file descriptors, fd[0], fd[1]
// fd[0] is for reading, fd[1] is for writing
// Anything that is written on fd[1] can be read from fd[0].
…
}
 a file descriptor is an index for an entry in a kernel data structure
containing details of all open files.
 Pipes are treated as files
 You can use system calls read(), write(), close() … on pipes
Simple example
#include <stdio.h>
main() {
int pipefd[2];
int i;
char s[1000];
char *s2;
if (pipe(pipefd) < 0) {
perror("pipe"); write() to a write end of a pipe: asks
exit(1); operating system to hold those bytes in a
} buffer until some process requests for them
by performing a read() on the read end of
s2 = “Hello World!"; the pipe.
write(pipefd[1], s2, strlen(s2));
i = read(pipefd[0], s, 1000);
s[i] = '\0';
printf("Read %d bytes from the pipe: '%s'\n", i, s);
}
Visualize pipes

program
File
Code, descriptors
Globals
stacks 0
1
2 Operating buffer
pipefd[0] system
pipefd[1] Hello World!

heap
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <string.h>
#define BUFFER_SIZE 25
#define READ_END 0
#define WRITE_END 1
int main(void)
{
char write_msg[BUFFER_SIZE] = "Greetings";
char read_msg[BUFFER_SIZE];
pid_t pid;
int fd[2];
/** create the pipe */
if (pipe(fd) == -1) {
fprintf(stderr,"Pipe failed");
return 1;
}
/** now fork a child process */
pid = fork();
if (pid < 0) {
fprintf(stderr, "Fork failed");
return 1;
}
 if (pid > 0) { /* parent process */
/* close the unused end of the pipe */
close(fd[READ_END]);
/* write to the pipe */
write(fd[WRITE_END], write_msg, strlen(write_msg)+1);
/* close the write end of the pipe */
close(fd[WRITE_END]);
}
else { /* child process */
/* close the unused end of the pipe */
close(fd[WRITE_END]);
/* read from the pipe */
read(fd[READ_END], read_msg, BUFFER_SIZE);
printf("child read %s\n",read_msg);
/* close the write end of the pipe */
close(fd[READ_END]);
}
return 0;
}
 dup, dup2 system calls
 How to implement command line pipeline, or standard
input/output redirection?
 dup and dup2 create a copy of file descriptor oldfd.
 Old and new descriptors may be used interchangeably, and they share
locks, file position pointers and flags.
int dup(int oldfd);
int dup2(int oldfd, int newfd);
 dup uses lowest-numbered unused descriptor for new descriptor
 dup2 makes newfd be copy of oldfd, closing newfd first if
necessary.

30 CSRU 3130: Unix System Programming

Code studies:
Command Line Pipeline
Communications for Distributed Systems
 Distributed Systems (Computing Environment)
 Client-Server Computing: computing server, file server, web
server ….
 Peer-to-peer systems: all nodes are both client and server;or
sometimes acts as client, sometimes as server
 Building distributed systems involves communications
between processes running on different systems
 Sockets
 Remote Procedure Calls
 Remote Method Invocation (Java)

32
Sockets
 A socket is defined as an endpoint for communication
 Identified by IP address and port
 With IP address, networking protocol routes the packet to the
destined host
 Different servers listens for requests on well known port #
 E.g. the socket 161.25.19.8:1625 refers to port 1625 on
host 161.25.19.8
 Communication consists between a pair of sockets
 TCP socket: reliable stream based communication
 UDP socket: unreliable datagram communication

33
Socket Communication

34
Remote Procedure Calls
• Remote procedure call (RPC) abstracts procedure calls
between processes on networked systems
• Stubs – client-side proxy for the actual procedure on the
server
• Client-side stub locates server and marshalls parameters
• Parameter marshalling: packaging parameters into a form to be
transmitted over network
• Server-side stub (daemon) receives this message, unpacks
marshalled parameters, and performs requested procedure
on server
• Value is returned (if needed) to client in similar way

35
Marshalling Parameters

36
Execution of RPC

37
Examples of IPC Systems – Windows XP
• Message-passing centric via local procedure call (LPC)
facility
– Only works between processes on the same system
– Uses ports (like mailboxes) to establish and maintain
communication channels
– Communication works as follows:
• The client opens a handle to the subsystem’s connection port object
• The client sends a connection request
• The server creates two private communication ports and returns the
handle to one of them to the client
• The client and server use the corresponding port handle to send
messages or callbacks and to listen for replies

38
Local Procedure Calls in Windows XP

39
Remote Method Invocation
 Remote Method Invocation (RMI) is a Java
mechanism similar to RPCs
 RMI allows a Java program on one machine to invoke
a method on a remote object

40
File Access: typical use
• #include <stdio.h>
• FILE *fp;
• To open a file
– fp = fopen(“~/tmp.txt”, “r+”);
• To read a character from a file:
– int c; c = getc(fp);
• To write a character to a file:
– putc(c, fp);
• Read/write using fscanf/fprintf, fgets/fputs, ..
• Finally, fclose(fp) is used to close a file
41 CSRU3130 Unix Programming Spring 2008
fopen() routine (1)
#include <stdio.h>
FILE *fopen(const char *path, const char *mode);
• path: a relative or full path name of the file
• mode: access mode
– “r”: open the file to read
– “w”: open the file to write (existing content will be discarded)
– “a”: open file to append (write starts at end of file)
– “rw”: open file to read and write
– …

42 CSRU3130 Unix Programming Spring 2008

fopen routine (2)
FILE *fopen(const char *path, const char *mode);
 FILE : a data structure containing info. needed to
perform input or output operations on it, including:
 a file descriptor (will study in low-level file access)
 current stream position
 an end-of-file indicator , an error indicator
 a pointer to the stream's buffer, if applicable
 Note:
 read/write: at current stream position
 Buffered I/O: not every write is applied to the disk
immediately
43 CSRU3130 Unix Programming Spring 2008
Example
• FILE * fp = fopen(“~/tmp.txt”, “r+”);
– takes a filename, does some housekeeping and negotiation with
the kernel
– Returns pointer to the FILE data structure on success; return
NULL on failure
• Always check for error after the call
if (fp==NULL){
printf (“failed to open file ~/tmp.txt\n”);
exit(1);
}

44 CSRU3130 Unix Programming Spring 2008

Meaningful error message
• errno – integer variable, set by system calls and some
library functions in event of an error to indicate what
went wrong
# include <errno.h>
…
if (fp==NULL){
switch (errno) {
Use “man errno” to find out
case EACCES: all error code.
printf (“You don’t have permission\n”);
break;
…
default:
printf (“Something went wrong in fopen\n”);
} exit(1); }

45 CSRU3130 Unix Programming Spring 2008

Using perror()
#include <stdio.h>
void perror(const char *s);
 perror() produces a message on standard error output,
describing the last error encountered, i.e., errno.
FILE * fp = fopen(“~/tmp.txt”, “r+”);
if (fp==NULL){
perror(“open ~/tmp.txt”);
exit(1);
}

46 CSRU3130 Unix Programming Spring 2008

Three special files
 Whenever a program is started, three files are
automatically opened, with file pointers stdin, stdout,
stderr.
 getchar() is same as getc(stdin)
 putchar(c) is same as putc(stdout, c).
 printf(s,…) is same as fprintf (stdout,s,…);
 scanf(s,…) is same as fscanf (stdin,s,…);

47 CSRU3130 Unix Programming Spring 2008

Other standard I/O Functions
 feof(FILE *): return non-zero when end of file is reached
 ferror(FILE *): return non-zero when any error
 fflush (FILE *): flush any buffered output to the file

48 CSRU3130 Unix Programming Spring 2008

New vis: handling files
int main(int argc, char *argv[]) {
int strip = 0;
int i;
FILE *fp;
while (argc > 1 && argv[1][0] == '-'){
switch (argv[1][1]) {
case 's': /* -s: strip funny characters */
strip = 1;
break;
default:
fprintf(stderr, "%s: unknown arg %s\n", argv[0], argv[1]);
return 1;
}
argc--; argv++;
}

49 CSRU3130 Unix Programming Spring 2008

main(): cont’d
if (argc == 1)
vis(stdin, strip);
else for (i = 1; i < argc; i++)
if ((fp = fopen(argv[i], "r")) == NULL) {
fprintf(stderr, "%s: can't open %s\n", argv[0], argv[1]);
return 1;
}
else {
vis(fp, strip);
fclose(fp);
}
return 0;
}50 CSRU3130 Unix Programming Spring 2008
Now vis
void vis(FILE *fp, int strip) {
int c;
while ((c = getc(fp)) != EOF)
if (isprint(c) || isspace(c))
putchar(c);
else if (!strip)
printf("\\%03o", c);
}

51 CSRU3130 Unix Programming Spring 2008