Processes, Threads and Processors

Hardware can interpret N instruction streams at once

¾ Uniprocessor, N==1
¾ Dual
Dual-core,
core, N==2
N 2
¾ Sun’s Niagara T2 (2007) N == 64, but 8 groups of 8
An OS can run 1 process on each processor at the
same time
From Processes to Threads ¾ Concurrent execution increases perforamnce
An OS can run 1 thread on each processor at the
same time

1 2
Processes and Threads The Case for Threads

Process abstraction combines two concepts Consider the following code fragment
¾ Concurrency
for(k = 0; k < n; k++)
™ Each process is a sequential execution stream of instructions
¾ Protection a[k] = b[k] * c[k] + d[k] * e[k];
™ Each process defines an address space
™ Address space identifies all addresses that can be touched by the Is there a missed opportunity here? On a Uni-processor?
program
On a Multi-processor?
Threads
¾ Key idea: separate the concepts of concurrency from protection
¾ A thread is a sequential execution stream of instructions
¾ A process defines the address space that may be shared by
multiple threads
¾ Threads can execute on different cores on a multicore CPU
(parallelism for performance) and can communicate with other
threads by updating memory

3 4
The Case for Threads Programmer’s View

Consider a Web server void fn1(int arg0, int arg1, …) {…}

get network message (URL) from client
get URL data from disk main()
i () {
compose response …
send response tid = CreateThread(fn1, arg0, arg1, …);
…
How well does this web server perform? }

At the point CreateThread is called

called, execution continues in parent
thread in main function, and execution starts at fn1 in the child
thread, both in parallel (concurrently)

5 6
Introducing Threads

A thread represents an abstract entity that executes a sequence

of instructions Context switch time for which entity is greater?
¾ It has its own set of CPU registers
¾ It has its own stack
¾ There is no thread-specific heap or data segment (unlike process) 1. Process
Threads are lightweight
2. Thread
¾ Creating a thread more efficient than creating a process.
¾ Communication between threads easier than btw. processes.
¾ Context switching between threads requires fewer CPU cycles and
memory references than switching processes.
¾ Threads only track a subset of process state (share list of open
files, pid, …)

Examples:
¾ OS-supported: Windows’ threads, Sun’s LWP, POSIX threads
¾ Language-supported: Modula-3, Java
™ These are possibly going the way of the Dodo

7 8
How Can it Help? How Can it Help?

How can this code take advantage of 2 threads? Consider a Web server
for(k = 0; k < n; k++) Create a number of threads, and for each thread do
a[k] = b[k] * c[k] + d[k] * e[k]; ™ get network message from client
™ get URL data from disk
Rewrite this code fragment as: ™ send data over network
do_mult(l, m) {
for(k = l; k < m; k++) What did we gain?
a[k] = b[k] * c[k] + d[k] * e[k];
}
main() {
CreateThread(do_mult, 0, n/2);
CreateThread(do_mult, n/2, n);

What did we gain?

9 10
Overlapping Requests (Concurrency)

Request 1 Request 2
Thread 1 Thread 2 Threads have their own…?
™ get network message
(URL) from client 1. CPU
™ get network message
™ get URL data from disk
(URL) from client 2. Address space
™ get URL data from disk 3. PCB
(disk access latency)
4. Stack
(disk access latency)
5. Registers
™ send data over network
™ dd
send k
data over network

Total time is less than request 1 + request 2

Time

11 12
 Threads vs. Processes Implementing Threads
Process’s
Threads Processes TCB for address space
Processes define an address Thread1
space; threads share the mapped segments
A thread has no data segment A process has code/data/heap & address space PC
or heap th segments
other t SP DLL’ss
DLL
A thread cannot live on its own, There must be at least one Process Control Block (PCB) State
it must live within a process Registers Heap
thread in a process contains process-specific
There can be more than one …
Threads within a process share information
thread in a process, the first ¾ Owner, PID, heap pointer,
thread calls main & has the code/data/heap, share I/O, but
each has its own stack & priority, active thread, and
process’s stack pointers to thread information TCB for Stack – thread2
registers
If a thread dies, its stack is Thread2
reclaimed If a process dies, its resources
Thread Control Block (TCB)
Inter-thread communication via are reclaimed & all threads die PC Stack – thread1
contains thread-specific SP
memory. Inter-process communication via information State
Each thread can run on a OS and data copying. Initialized data
different physical processor
¾ Stack pointer, PC, thread state Registers
Each process can run on a (running, …), register values, a …
Inexpensive creation and different physical processor pointer to PCB, … Code
context switch
Expensive creation and context
switch
13 14
Threads’ Life Cycle

Threads (just like processes) go through a sequence of start, Threads have the same scheduling states as
ready, running, waiting, and done states processes

1. True
T
Start Done 2. False

Ready Running

Waiting

15 16
User­level vs. Kernel­level threads
Process0
User-level threads (M to 1 model)
¾ + Fast to create and switch
¾ + Natural fit for language-level threads
¾ - All user-level threads in process block on OS calls
™ E.g., read from file can block all threads
¾ -User-level scheduler can fight with kernel-level scheduler
Kernel-level threads (1 to 1 model)
¾ + Kernel-level threads do not block process for syscall
¾ + Only one scheduler (and kernel has global view)
¾ - Can be difficult to make efficient (create & switch)
Languages vs. Systems
Process1
user Kernel-level threads have won for systems
kernel ¾ Linux, Solaris 10, Windows
¾ pthreads tends to be kernel-level threads
User-level threads still used for languages (Java)
¾ User tells JVM how many underlying system threads
™ Default: 1 system thread
¾ Java runtime intercepts blocking calls, makes them non-
blocking
¾ JNI code that makes blocking syscalls can block JVM
¾ JVMs are phasing this out because kernel threads are
efficient enough and intercepting system calls is complicated
Kernel-level thread vs. process
¾ Each process requires its own page table & hardware state
(significant on the x86)
17 18
Latency and Throughput
Latency: time to complete an operation
Throughput: work completed per unit time
Multiplying vector example: reduced latency
Web server example: increased throughput
Consider plumbing
¾ Low latency: turn on faucet and water comes out
¾ High bandwidth: lots of water (e.g., to fill a pool)
What is “High speed Internet?”
¾ Low latency: needed to interactive gaming
¾ High bandwidth: needed for downloading large files
¾ Marketing departments like to conflate latency and
bandwidth…
19
Relationship between Latency and Throughput
Latency and bandwidth only loosely coupled
¾ Henry Ford: assembly lines increase bandwidth without
reducing latency
My factory takes 1 day to make a Model-T ford.
¾ But I can start building a new car every 10 minutes
¾ At 24 hrs/day, I can make 24 * 6 = 144 cars per day
¾ A special order for 1 green car, still takes 1 day
¾ Throughput is increased, but latency is not.
Latency reduction is difficult
Often, one can buy bandwidth
¾ E.g., more memory chips, more disks, more computers
¾ Big server farms (e.g., google) are high bandwidth
20
Thread or Process Pool
When a user level thread does I/O it blocks the
Creating a thread or process entire process.
for each unit of work (e.g., Well conditioned
q
user request)) is dangerous
g Not well conditioned
¾ High overhead to create &
delete thread/process 1. True
¾ Can exhaust CPU & 2. False
Throughput

memory resource
Thread/process pool controls
resource use
¾ Allows service to be well
conditioned.

Load

21 22