ארכיטקטורה - הרצאה 5 - Virtual Memory and Memory Hierarchy

Processes, Interrupts, and Exceptions
Based on slides by C. Kozyrakis
Memory manager
Processes, Exceptions, Interrupts, and Virtual Memory

Operating system Manages hardware resources (CPU, memory, I/O devices)
On behalf of user applications
Provides protection and isolation Virtualization

Processes and virtual memory
What hardware support is required for the OS? Kernel and user mode
Kernel mode: access to all state including privileged state User mode: access to user state ONLY
Exceptions/interrupts Virtual memory support
Memory manager
Processes
Definition: A process is an instance of a running program. Virtualization of CPU and memory Process provides each program with two key abstractions: Logical control flow
Gives each program the illusion that it has exclusive use of the CPU Private set of register values
Private address space

Gives each program the illusion that has exclusive use of main memory of infinite size
How is this illusion maintained? Process execution is interleaved (multitasking) Address space is managed by virtual memory system
Memory manager
Logical Control Flows

Each process has its own logical control flow
Process A Time Process B Process C
Memory manager
Concurrent Processes
Two processes run concurrently (are concurrent) if their flows overlap in time. Otherwise, they are sequential. Examples: Concurrent: A & B, A&C Sequential: B & C
Process A Time Process B Process C
Memory manager
User view of concurrent processes

Control flows for concurrent processes are physically disjoint in time However, we can think of concurrent processes are running in parallel with each other
Process A Time
Process B
Process C
Memory manager
Context Switching
Processes are managed by a shared chunk of OS code called the kernel Important: the kernel is not a separate process, but rather runs as part of some user process Control flow passes from one process to another via a context switch Time between context switches 1020 ms Process A code Process B code
user code context switch
Time
kernel code user code kernel code user code
context switch
Memory manager
Process Address Space

0xffffffff kernel memory (code, data, heap, stack) 0x80000000 user stack (created at runtime) memory invisible to user code $sp (stack pointer)
Each process has its own private address space User processes cannot access top region of memory
run-time heap (managed by malloc) 0x10000000 0x00400000 0 read/write segment (.data, .bss) read-only segment (.text) unused
brk
loaded from the executable file
Memory manager
Altering the Control Flow

Weve discussed two mechanisms for changing the control flow: Jumps and branches Call and return Both implement expected changes in program state System call instruction (syscall) A jump to operating system code
If user process decides to request kernel services
Switches mode from user to kernel Jumps to a pre-defined place in the kernel program Still an expected change
Memory manager
Unexpected Changes in Control Flow

Difficult for the CPU to react to unexpected changes in system state Data arrives from a disk or a network adapter Instruction divides by zero User hits ctl-c at the keyboard System timer expires CPU needs mechanisms for exceptional control flow
Memory manager
10
Exceptions and Interrupts and Traps TRAPS

Change in control flow in response to a system event Switch CPU mode from user to kernel Interrupts External CPU events I/O device (disk, keyboard) Clock Exceptions Internal CPU events Arithmetic overflow Illegal instruction Virtual memory address exception Syscall can be viewed as an exceptional event as well Implemented as a combination of both hardware and OS software
Memory manager
11
Exceptional Control Flow

An exception is a transfer of control to the OS in response to some event User Process OS
event
current next
exception exception processing by exception handler exception return (optional)
Precise exceptions/interrupts Difficult for deeply pipelined processors Could get exceptions out of order
Memory manager
12
Reminder: Precise Exceptions

Definition: All instructions before one that causes exception are fully done The faulting instruction and all that follow do not change program state
As if they never executed at all
Why precise exceptions are difficult with pipelining Imagine a 5 state pipeline during cycle i
Stage MEM: a ld causes a virtual memory exception Stage EX: an add causes an arithmetic overflow Stage ID: an invalid instruction exceptions Stage IF: a virtual memory exceptions due to instruction fetch
Concurrent or out-of-order exceptions!
Memory manager
13
Precise Exceptions for Pipelined Processors

Solution: defer exception handling until WB stage If instruction raised in previous stage, note it in pipeline register, and turn instruction to nop When instruction reaches WB stage: flush pipeline & raise exception This works because: Instructions go to WB stage one at the time Instructions go to the WB stage in order External events (e.g. IO interrupts) Attach them to a random instruction & proceed as shown above
Memory manager
14
MIPS Interrupts
What does the CPU do on an exception? Set EPC register to point to the restart location Change CPU mode to kernel and disable interrupts Set Cause register to reason for exception also set BadVaddr register if address exception Jump to interrupt vector Interrupt Vectors 0x8000 0000 : TLB refill 0x8000 0180 : All other exceptions Privileged state EPC Cause BadVaddr
Memory manager
15
A Really Simple Exception Handler

xcpt_cnt: la lw addu sw eret
$k0, $k1, $k1, $k1,
xcptcount 0($k0) 1 0($k0)
# # # # # # #
get address of counter load counter increment counter store counter restore status reg: enable interrupts and user mode return to program (EPC)
Cant survive nested exceptions OK since does not re-enable interrupts Does not use any user registers No need to save them
Memory manager
16
Interrupt vectors Accelerating handler dispatch

Exception numbers code for code for exception handler 00 exception handler code for code for exception handler 11 exception handler code for code for exception handler 22 exception handler 1. Each type of interrupt has a unique exception number k 2. Jump table (interrupt vector) entry k points to a function (exception handler). 3. Handler k is called each time exception k occurs
interrupt vector
0 1 2 n-1
...
...
code for code for exception handler n-1 exception handler n-1 But the CPU is much faster than I/O devices and most OSs use a common handler anyway Make the common case fast!
Memory manager
17
Exception Example #1
Memory Reference User writes to memory location That portion (page) of users memory is currently not in memory Page handler must load page into physical memory Returns to faulting instruction Successful on second try User Process OS
int a[1000]; main () { a[500] = 13; }
event
lw
page fault return Create page and load into memory
Memory manager
18
Exception Example #2
Memory Reference User writes to memory location Address is not valid Page handler detects invalid address Sends SIGSEG signal to user process User process exits with segmentation fault
int a[1000]; main () { a[5000] = 13; }
User Process
OS
event
lw
page fault Detect invalid address Signal process
Memory manager
19
Review: OS User Code Switches

System call instructions (syscall) Voluntary transfer to OS code to open a file, write data, change working directory, Jumps to predefined location(s) in the operating system Automatically switches from user mode to kernel mode
PC + 4
Exceptions & interrupts Illegal opcode, device by zero, , timer expiration, I/O, Exceptional instruction becomes a jump to OS code
Precise exceptions
Branch/jump PC PC Excep Vec MUX EPC
External interrupts attached to some instruction in the pipeline New state: EPC, Cause, BadVaddr registers
Memory manager
20

0xffffffff kernel memory (code, data, heap, stack) 0x80000000 user stack (created at runtime) memory invisible to user code $sp (stack pointer)
Each process has its own private address space
run-time heap (managed by malloc) 0x10000000 0x00400000 0 read/write segment (.data, .bss) read-only segment (.text) unused
brk
loaded from the executable file
Memory manager
21
Motivations for Virtual Memory

#1 Use Physical DRAM as a Cache for the Disk Address space of a process can exceed physical memory size Sum of address spaces of multiple processes can exceed physical memory #2 Simplify Memory Management Multiple processes resident in main memory.
Each process with its own address space
Only active code and data is actually in memory

Allocate more memory to process as needed.
#3 Provide Protection One process cant interfere with another.

because they operate in different address spaces.
User process cannot access privileged information

different sections of address spaces have different permissions.
Memory manager
22
Motivation #1: DRAM a Cache for Disk

Full address space is quite large:
32-bit addresses: ~4,000,000,000 (4 billion) bytes 64-bit addresses: ~16,000,000,000,000,000,000 (16 quintillion) bytes
Disk storage is ~100X cheaper than DRAM storage To access large amounts of data in a cost-effective manner, the bulk of the data must be stored on disk
1 GB: ~$128 4 MB: ~$120 SRAM DRAM
100 GB: ~$125
Disk
Memory manager
23
Levels in Memory Hierarchy

cache virtual memory
CPU CPU regs regs Register size: 128 B Speed(cycles): 0.5-1 $/Mbyte: line size: 8B
8B
C a c h e
32 B
Memory Memory
8 KB
disk disk
Cache <4MB 1-20 $30/MB 32 B
Memory < 16 GB 80-100 $0.128/MB 8 KB
Disk Memory > 100 GB 5-10 M $0.001/MB
larger, slower, cheaper
Memory manager
24
DRAM vs. SRAM as a Cache

DRAM vs. disk is more extreme than SRAM vs. DRAM Access latencies:
DRAM ~10X slower than SRAM Disk ~100,000X slower than DRAM
Importance of exploiting spatial locality:

First byte is ~100,000X slower than successive bytes on disk
vs. ~4X improvement for page-mode vs. regular accesses to DRAM
Bottom line:
Design decisions made for DRAM caches driven by enormous cost of misses
SRAM
DRAM
Disk
Memory manager
25
Impact of These Properties on Design

If DRAM was to be organized similar to an SRAM cache, how would we set the following design parameters? Line size?
Large, since disk better at transferring large blocks
Associativity?
High, to mimimize miss rate
Write through or write back?

Write back, since cant afford to perform small writes to disk
What would the impact of these choices be on: miss rate

Extremely low. << 1%
hit time
Must match cache/DRAM performance
miss penalty
Very high. ~20ms
tag storage overhead

Low, relative to block size
Memory manager
26
Locating an Object in a Cache

SRAM Cache Tag stored with cache line Maps from cache block to memory blocks
From cached to uncached form
No tag for block not in cache Hardware retrieves information

can quickly match against multiple tags Cache Tag Object Name X 0: D X J Data 243 17 105
= X?
1: N-1:
Memory manager
27
Locating an Object in a Cache (cont.)

DRAM Cache Each allocate page of virtual memory has entry in page table Mapping from virtual pages to physical pages
From uncached form to cached form
Page table entry even if page not in memory

Specifies disk address
OS retrieves information
Page Table Location Object Name X D: J: X: 0
On Disk
Cache Data 0: 1: N-1: 243 17 105
Memory manager
28
A System with Physical Memory Only

Examples: Most early Cray machines, early PCs, nearly all embedded systems, etc. Memory Physical Addresses 0: 1:
CPU
N-1:
Addresses generated by the CPU point directly to bytes in physical memory

Based on slides by C. Kozyrakis Memory manager 29
A System with Virtual Memory

Examples: workstations, servers, modern PCs, etc. Page Table Virtual Addresses 0: 1: Physical Addresses Memory 0: 1:
CPU
P-1:
N-1: Disk
Address Translation: Hardware converts virtual addresses to physical addresses via an OS-managed lookup table (page table)
Page Faults (Similar to Cache Misses)

What if an object is on disk rather than in memory? Page table entry indicates virtual address not in memory OS exception handler invoked to move data from disk into memory
current process suspends, others can resume OS has full control over placement, etc.
Before fault
Memory Page Table Virtual Physical Addresses Addresses CPU
After fault
Memory Page Table Virtual Addresses CPU Physical Addresses
Disk
Disk
Memory manager
31
Servicing a Page Fault

(1) Initiate Block Read Processor Signals Controller Read block of length P starting at disk address X and store starting at memory address Y Read Occurs Direct Memory Access (DMA) Under control of I/O controller I / O Controller Signals Completion Interrupt processor OS resumes suspended process Processor Processor
Reg
(3) Read Done
Cache Cache Memory-I/O bus Memory-I/O bus (2) DMA Transfer Memory Memory I/O I/O controller controller
disk Disk
disk Disk
Memory manager
32
Motivation #2: Memory Management

Multiple processes can reside in physical memory. How do we resolve address conflicts? what if two processes access something at the same address?
kernel virtual memory stack memory invisible to user code
$sp
the brk ptr runtime heap (via malloc) uninitialized data (.bss) initialized data (.data) program text (.text) forbidden
Memory manager
33
Solution: Separate Virtual Addr. Spaces

Virtual and physical address spaces divided into equal-sized blocks
blocks are called pages (both virtual and physical)
Each process has its own virtual address space

operating system controls how virtual pages as assigned to physical memory
0
Virtual Address Space for Process 1:
0 VP 1 VP 2
Address Translation PP 2
...
Physical Address Space (DRAM)

(e.g., read/only library code)
N-1 PP 7
Virtual Address Space for Process 2:
VP 1 VP 2
...
PP 10 M-1
N-1
Memory manager
34
Motivation #3: Protection

Page table entry contains access rights information hardware enforces this protection (trap into OS if violation occurs) Page Tables
Read? Write? VP 0: Yes No Yes No Physical Addr PP 9 PP 4 XXXXXXX
Memory 0: 1:
Process i:
VP 1: Yes VP 2: No

Physical Addr PP 6 PP 9 XXXXXXX
Read? Write? VP 0: Yes Yes No No
Process j:
VP 1: Yes VP 2: No
N-1:

EE108b Lecture 13
Memory manager 35
Paging Address Translation
1K Page Table Lookup
1K
2048 1024 0
Page 2 Page 1 Page 0
1K 1K 1K
2048 1024 0
Frame 2 Frame 1 Frame 0
1K 1K 1K
Virtual Memory
Physical Memory
Virtual pages can map to any physical frame (fully associative)
Memory manager
36
Page Table/Translation
1 KB Pages
Virtual address
Page Table Base Register Valid bit to indicate if virtual page is currently mapped
22 10 Page number Offset
= 32
Page Table
Page table index
V Access Rights Frame
Access rights to define legal accesses (Read, Write, Execute)
Frame # 30 = 20
Physical address To memory
Offset 10
Table located in physical memory
Memory manager
37

Each process has its own address space Each process must therefore have its own translation table Translation tables are changed only by the OS A process cannot gain access to another processs address space Context switches Steps
1. 2. 3. 4. Save previous process state (registers, PC) in the Process Control Block (PCB) Change page table pointer to new processs table Flush the TLB (discussed later) Initialize PC and registers from new process PCB
Context switches occur on a timeslice (when the scheduler determines the next process should get the CPU) or when blocking for I/O
Memory manager
38
Translation Process
Valid page
Check access rights (R, W, X) against access type
Generate physical address if allowed Generate a protection fault if illegal access
Invalid page
Page is not currently mapped and a page fault is generated
Faults are handled by the operating system

Protection fault is often a program error and the program should be terminated Page fault requires that a new frame be allocated, the page entry is marked valid, and the instruction restarted
Frame table has mapping from physical address to virtual address and tracks used frames
Memory manager
39
Unaligned Memory Access

Memory access might be aligned or unaligned 0 0 4 4 8 8 12 12 16 16
What happens if unaligned address access straddles a page boundary?

What if one page is present and the other is not? Or, what if neither is present?
MIPS architecture disallows unaligned memory access Interesting legacy problem on 80x86 which does support unaligned access
Memory manager
40
Page Table Problems

Internal Fragmentation resulting from fixed size pages since not all of page will be filled with data
Problem gets worse as page size is increased
Each data access now takes two memory accesses

First memory reference uses the page table base register to lookup the frame Second memory access to actually fetch the value
Page tables are large

Consider the case where the machine offers a 32 bit address space and uses 4 KB pages
Page table contains 232 / 212 = 220 entries = 1M Assume each entry contains ~32 bits = 4 Byte RAM 4 MB of Page table requires
per
process!
Memory manager
41
Improving Access Time

But we know how to fix this problem, lets cache the page table and leverage the principles of locality Rather than use the normal caches, we will use a special cache known as the Translation Lookaside Buffer (TLB)
All accesses are looked up in the TLB A hit provides the physical frame number On a miss, look in the page table for the frame TLBs are usually smaller than the cache Higher associativity is common Similar speed to a cache access Treat like a normal cache and use write back vs. write through
Memory manager
42
TLB Entries
Just holds a cached Page Table Entry (PTE) Additional bits for LRU, etc. may be added
Virtual Page Physical Frame Dirty LRU Valid Access Rights
Optimization is to access TLB and cache in parallel

Virtual Addr CPU TLB Lookup miss map entry hit Phy Addr Cache hit data miss Main Memory
Access page tables for translation (from memory!)
data
Memory manager
43
Address Translation with a TLB

n1 p p1 0 virtual page number page offset virtual address
valid
tag physical page number
.
=
TLB
TLB hit
physical address tag valid tag data index byte offset
Cache
cache hit
data
Memory manager
44
TLB Miss Handler

A TLB miss can be handled by software or hardware Miss rate : 0.01% 1% In software, a special OS handler looks up the value in the page table Ten instructions on R3000/R4000 In hardware, microcode or a dedicated finite state machine (FSM) looks up the value Page tables are stored in regular physical memory It is therefore likely that the data is in the L2 cache Advantages
No need to take an exception Better performance but may be dominated by cache miss
Memory manager
45
TLB problems
What happens on a context switch?
If the TLB entries have a Process ID (PID) associated with them, then nothing needs to be done Otherwise, the OS must flush the entries in the TLB
Limited Reach
64 entry TLB with 8KB pages maps 0.5 MB Smaller than many L2 caches TLB miss rate > L2 miss rate! Motivates larger page size
Memory manager
46
TLB Case Study: MIPS R2000/R3000

Consider the MIPS R2000/R3000 processors
Addresses are 32 bits with 4 KB pages (12 bit offset) TLB has 64 entries, fully associative Each entry is 64 bits wide:
20 6 6 20 1 1 1 1 8
Virtual Page PID 0 Physical Page N D V G 0 PID N D V G Process ID Do not cache memory address Dirty bit Valid bit Global (valid regardless of PID)
Memory manager
47
MIPS UTLB Miss Handler in Software

In software, a special OS handler looks up the value in the page table UTLBmiss: mfc0 $k1, Context # copy address of PTE to $k1 lw $k1, 0($k1) # put PTE into $k1 mtc0 $k1, Entrylo # put PTE into CP0 Entrylo tlbwr # put Entryhi, Entrylo into # TLB at Random eret # return from UTLB miss
Memory manager
48
Page Sharing
Another benefit of paging is that we can easily share pages by mapping multiple pages to the same physical frame Useful in many different circumstances
Useful for applications that need to share data Read only data for applications, OS, etc. Example: Unix fork() system call creates second process with a copy of the current address space
Often implemented using copy-on-write

Processes have read-only access to the page If any process wants to write, then a page fault occurs and the page is copied
Memory manager
49
Page Sharing (cont)
Virtual Address
1K 0 Virtual Memory Process 1
Physical Address 1K 0
Physical Memory
Alias: Two VAs that map to same PA
Memory manager
50
Copy-on-Write
Virtual Address
Physical Address 1K 1K 0 Physical Memory
Memory manager
51
Page Table Size

Page table size is proportional to size of address space Even when applications use few pages, page table is still very large Example: Intel 80x86 Page Tables Virtual addresses are 32 bits Page size is 4 KB Total number of pages: 232 / 212 = 1 Million Page Table Entry (PTE) are 32 bits wide
20 bit Frame address Dirty bit, accessed bit, valid (present) bit 9 flag and unused bits
Total page table size is therefore 220 4 bytes = 4 MB But, only a small fraction of those pages are actually used! The large page table size is made even worse by the fact that the page table must be in memory Otherwise, what would happen if the page table were suddenly swapped out?
Memory manager
52
Paging the Page Tables

One solution is to use multi-level page tables If each PTE is 4 bytes, then each 4 KB frame can hold 1024 PTEs Use an additional master block 4 KB frame to index frames containing PTEs
Master frame has 1024 PFEs (Page Frame Entries) Each PFE points to a frame containing 1 KB PTEs Each PTE points to a page
Now, only the master block must be fixed in memory
Memory manager
53
Two-level Paging
PFE PTR PTE Offset
+ +
Master Block
+
Page Table Desired Page
Problems
Multiple page faults
Accessing a PTE page table can cause a page fault Accessing the actual page can cause a second page fault
TLB plays an even more important role
Memory manager
54
Virtual Memory and Caches: TLB and Cache in Parallel

virtual page no. page offset
x de g n l i l ta a a r t u si c vi hy p
TLB
Cache Tag check
Main memory
data
19b 13b virtual page number page offset cache tag 19b cache byte index offset 9b 4b
Translation & cache access in 1 cycle Access cache with untranslated part of address Tag check used physical address Cache data is based on physical addresses No problems with aliasing, I/O, multiprocessor snooping But only works when VPN bits not needed for cache lookup Cache Size <= Page Size * Assoc i.e. Set Size <= Page Size Most common solution We want L1 to be small anyway
Memory manager
55
Virtual Memory and Caches

TLB hit miss miss miss hit hit miss Cache miss hit miss miss miss hit hit Virtual memory hit hit hit miss miss miss miss Possible? yes yes yes yes no no no
Memory manager
56
Virtual Memory Overview

Presents the illusion of a large address space to every application
Each process or program believes it has its own complete copy of physical memory (its own copy of the address space)
Strategy
Pages not currently in use can be written back to secondary storage When page ownership changes, save the current contents to disk so they can be restored later
Costs
Address translation is still required to convert virtual addresses into physical addresses Writing to the disk is a slow operation TLB misses can limit performance
Memory manager
57
Review: Memory Hierarchy Framework

Placing a Block
Direct mapped Element hashed to a single location Set Associative Element can go in one of N locations Fully Associative Element can go anywhere
Finding a Block
Index with partial map (cache) or full map (page table) Index and search (set associative) Search (fully associative)
Replacing a Block
Random Pick one of the elements to replace Least Recently Used (LRU) Use bits to track usage
Writing
Write back Data is written only on eviction Write through Each write is passed through to lower level
Locality makes this work

Five Components
Computer Processor Control Datapath Memory Devices Input Output
Datapath Control Memory Input Output
Memory manager
59

ארכיטקטורה - הרצאה 5 - Virtual Memory and Memory Hierarchy

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ארכיטקטורה - הרצאה 5 - Virtual Memory and Memory Hierarchy

Uploaded by

Copyright:

Available Formats

Processes, Interrupts, and Exceptions

Based on slides by C. Kozyrakis

Based on slides by C. Kozyrakis

Processes, Exceptions, Interrupts, and Virtual Memory

Provides protection and isolation Virtualization

Exceptions/interrupts Virtual memory support

Based on slides by C. Kozyrakis

Private address space

Based on slides by C. Kozyrakis

Logical Control Flows

Based on slides by C. Kozyrakis

Based on slides by C. Kozyrakis

User view of concurrent processes

Based on slides by C. Kozyrakis

kernel code user code kernel code user code

Based on slides by C. Kozyrakis

Process Address Space

loaded from the executable file

Based on slides by C. Kozyrakis

Altering the Control Flow

Based on slides by C. Kozyrakis

Unexpected Changes in Control Flow

Based on slides by C. Kozyrakis

Exceptions and Interrupts and Traps TRAPS

Based on slides by C. Kozyrakis

Exceptional Control Flow

exception exception processing by exception handler exception return (optional)

Based on slides by C. Kozyrakis

Reminder: Precise Exceptions

Concurrent or out-of-order exceptions!

Based on slides by C. Kozyrakis

Precise Exceptions for Pipelined Processors

Based on slides by C. Kozyrakis

Based on slides by C. Kozyrakis

A Really Simple Exception Handler

$k0, $k1, $k1, $k1,

xcptcount 0($k0) 1 0($k0)

Based on slides by C. Kozyrakis

Interrupt vectors Accelerating handler dispatch

Based on slides by C. Kozyrakis

int a[1000]; main () { a[500] = 13; }

page fault return Create page and load into memory

Based on slides by C. Kozyrakis

int a[1000]; main () { a[5000] = 13; }

page fault Detect invalid address Signal process

Based on slides by C. Kozyrakis

Review: OS User Code Switches

Branch/jump PC PC Excep Vec MUX EPC

Based on slides by C. Kozyrakis

Process Address Space

Each process has its own private address space

loaded from the executable file

Based on slides by C. Kozyrakis

Motivations for Virtual Memory

Only active code and data is actually in memory

#3 Provide Protection One process cant interfere with another.

User process cannot access privileged information

Based on slides by C. Kozyrakis

Motivation #1: DRAM a Cache for Disk

1 GB: ~$128 4 MB: ~$120 SRAM DRAM

100 GB: ~$125

Based on slides by C. Kozyrakis