Lect 03 PDF

LECTURE 3: REVIEW OF
MEMORY ORGANIZATION AND

HIERARCHY
Dr. ASSAF M. H.
EE326 - EMBEDDED SYSTEMS
USP JULY 2018

TOPICS
• Typical Memory Hierarchy
• Memory Hierarchy Parameters
• Memory Properties
• Hit Ratio
• Access Frequency
• Access Time
• Cache, Main, Virtual Memory, ROM

A Typical Memory Hierarchy
Memory in a computer system is usually characterized as
appearing at various hierarchical levels, with level 0 being
CPU registers, level 1 being the cache closest to the CPU,
and level 4 being the magnetic tape units representing the
lowest hierarchy level
Memory Hierarchy Parameters
The memory hierarchy can be characterized by a
number of parameters
Among these parameters are:
 access type
 capacity
 cycle time
 latency
 bandwidth
 cost
Fundamentals of Computer Organization and Architecture Mostafa Abd-El-Barr & Hesham El-Rewini
Memory Generalities
Each memory level is characterized:
 access time ti ( round-trip time from CPU to the ith level
memory )
 memory size si ( number of bytes or words in level i )
 cost per byte ci ( cost of the ith level memory is estimated
by ci x si )
 transfer bandwidth bi ( rate of transfer between adjacent levels )
 unit of transfer xi ( grain size for data transfers between adjacent
levels )
Memories at lower-numbered levels when compare to those at
higher-numbered levels:
 are faster to access
 are smaller in capacity
 are more expensive per byte
 have a higher bandwidth, and
 have a smaller unit of transfer
In general, then, ti-1 < ti, si-1 < si, ci-1 > ci, bi-1 > bi, and xi-1 < xi
for i = 1,2,3,and 4 where i = 0 corresponds to the CPU register
level
Information stored in a memory hierarchy (M1 M2 ,…, Mn)
satisfies three important properties: inclusion, coherence, and
The Inclusion Property
The inclusion property is stated as:
M1  M2  ...  Mn
The implication of the inclusion property is that all
items of information in the “innermost” memory level
(cache) also appear in the outer memory levels
The inverse, however, is not necessarily true. That is,

the presence of a data item in level Mi+1 does not
imply its presence in level Mi
We call a reference to a missing item a “miss”
A word miss in Mi implies that it is also missing from

all lower levels Mi-1, Mi-2, … ,M1
The Coherence Property
As information is modified by the processor, copies
of that information should be placed in the
appropriate locations in outer memory levels
The requirement that copies of data items at

successive memory levels be consistent is called the
“coherence property”
If a word is modified in the cache, copies of that

word must be updated at all higher levels
There are two strategies for maintaining the

coherence in the memory hierarchy
Coherence Strategies
Write-through (WT)
 As soon as a data item in Mi is modified,
immediate update of the corresponding data

item(s) in Mi+1, Mi+2, … Mn is required (
aggressive approach)
Write-back (WB)
 The update of the data item in Mi+1
corresponding to a modified item in Mi is not
updated until the word being modified in Mi is
replaced or moved from Mi ( most efficient
approach )
Hit Ratios
A hit represents a found item (instruction or data) in
the level of the memory hierarchy being examined
A miss represents a not found item (the item must

then be obtained from a lower level in the memory
hierarchy)
The hit ratio, hi, at Mi is the probability (between 0

and 1) that a needed information item is found when
sought in level memory Mi
The miss ratio at Mi is obviously just 1- hi
We assume h0 = 0 (CPU always accesses M1) and hn =

1 (the access to the outermost memory Mn is always a
hit)
Effectiveness of a Memory
Hierarchy
The sequence of events that takes place when the processor
makes a request for an item is as follows:
 First, the item is sought in the first memory level of the memory
hierarchy.
 The probability of finding the requested item in the first level is
called the hit ratio, h1
 The probability of not finding (missing) the requested item in the
first level of the memory hierarchy is called the miss ratio, (1- h1)
 When the requested item causes a “miss”, it is sought in the next
subsequent memory level
 The probability of finding the requested item in the second memory
level, the hit ratio of the second level, is h2
 The miss ratio of the second memory level is (1- h2)
 The process is repeated until the item is found. Upon finding the
requested item, it is brought and sent to the processor
Access Frequencies
The access frequency fi to level Mi is defined as:
fi = (1 - h1)  (1 - h2)  …  (1 - hi-1)  hi
It represents the probability of successfully

accessing Mi when there are i-1 misses at the lower
levels and a hit at Mi
n
Note that f1 = h1, and f

i 1
i 1
The access frequencies decrease from low to high

levels f1 >> f2 >> f3 >> … >> fn (the inner levels
of memory are accessed more often than the outer
levels)
Effective Access Times
Every time a miss occurs, a penalty must be paid to
access the next higher level in the memory hierarchy
 A block miss (misses in the cache) is typically 2 to 4
times as expensive as a cache hit
 A page fault (miss in the main memory) is 1000 to
10000 as costly as a page hit
The effective access time of a memory hierarchy can be

formally defined
n
as
Teff   fi  ti
i 1
 h1t1  (1  h1 )h2t2  L  (1  h1 )(1  h2 )L (1  hn 1 )hntn
* The first several terms in the above expression dominate,

but the effective access time is still dependent on the
program behavior and memory design choices
Cache Organization
Intel's Pentium IV Processor Cache
Main Memory
The main memory provides the main storage for a
computer
Two CPU registers are used to interface the CPU to
the main memory:
 The Memory Address Register (MAR) and
 The Memory Data Register (MDR): is used to hold the data
to be stored and/or retrieved in/from the memory location
whose address is held in the MAR
Typical Internal Main Memory
Structure
It is possible to visualize a typical internal main
memory structure as consisting of rows and
columns of basic cells
Each cell is capable of storing one bit of information
Main Memory Cell
In static CMOS technology, each main memory
cell consists of six transistors
The six transistor static CMOS memory cell
consists of two inverters back to back. It should
be noted that the cell could exist in one of the
two stable states
The two transistors N3 and N4 are used to
connect the cell to the two data (bit) lines
If the word select is not activated, these two
transistors are turned off, thus protecting the
cell from the signal values carried by the data
lines
The two transistors are turned on when the word
select line is activated. What takes place when
the two transistors are turned on will depend on
the intended memory operation
Static CMOS Memory Cell
Static CMOS Memory Operations
Read Operation
 Both lines b and are precharged high (1)
 The word select line is activated, thus turning on both
transistors N3 and N4
Write Operation
 Depending on the internal value stored in the cell, point
A (B) will lead to the discharge of line b ( )
 The bit lines are precharged such that b ( ) = 1 (0)
 The word select line is activated, thus turning both
transistors N3 and N4
 The bit line precharged with 0 will have to force the
point A (B), which is having 1, to 0
Internal Organization of a 1k x 4
Memory Chip
Different organization of the same memory capacity

can lead to different number of chip pins
requirements
Memory Sub-system
The available per chip memory capacity can be a

limiting factor in designing memory sub-systems
Organization of a 4M 8-bit Memory
Using 1M 1-bit memory Chips
The memory sub-system can be arranged in four
rows, each having eight chips
Dynamic Memory Cell
It is possible to use a 1-transistor dynamic cell

instead of the 6-transistor static random cell
Dynamic memory depends on storing logic values
using a capacitor together with one transistor
that acts as a switch
The use of dynamic memory leads to saving in
chip area
However, due to the possibility of decay of the
stored values (leakage of the stored charge in the
capacitor), dynamic memory requires periodical
(every few milliseconds) refreshment in order to
restore the stored logic values
Dynamic Memory Array
Organization
The read/write circuitry performs the functions
of sensing the value on the bit line, amplifying it,
and refreshing the value stored on the capacitor
Operation of Control Circuitry
CE R/W’ Operation
0 X None
1 1 Read
1 0 Write
Virtual Memory
The limited-size main memory (physical memory)
cannot load in all programs simultaneously
The virtual memory was introduced to solve this
problem
It expands the use of the physical memory among many
programs with the help of an auxiliary memory
Only active or portions of programs become residents of
the physical memory at one time
Inactive programs are stored in the auxiliary memory
Processors executing application programs normally
generate virtual addresses of data items and
instructions (not physical addresses)
Physical addresses are used to reference the available
locations in the real physical memory
Virtual addresses must be mapped to physical
addresses before they can be used
Virtual to Physical Mapping
The virtual to physical addresses mapping is
formally defined as follows:
 if m  M has been allocated to store

m,
ft v   the data identified by virtual address m

 if data v is missing in M
• The mapping returns a physical address (m) if a memory hit

occurs. When there is a memory miss, returned value (), the
referenced item has not yet been brought into the main
memory (M)
Memory Allocation
Both the virtual address space and the physical address
space are partitioned into fixed-length pages
 In the virtual address space these pieces are called pages
 In the physical address space they are called page frames
The purpose of memory allocation is to allocate pages

of virtual memory to the page frames of physical
memory
Memory Management Policies
• Include the allocation and deallocation of memory pages to active
processes and the replacement of memory pages
• The process in which resident page in main memory is replaced by
a new page transferred from the disk is called Page replacement
• R(t) defines the resident set of all pages residing in main memory at
time t
• Different page replacement policies:
– Least Recent Used (LRU): this policy replaces the page in R(t)
which has been least recently used
– First-in-first-out (FIFO): replaces the page in R(t) which has
been in memory for the longest time
– Least Frequently used (LFU): replaces the page in R(t) which
has been least referenced in the past
– Circular FIFO: joins all the page frame entries into a circular
FIFO queue
– Random Replacement (RR): chooses randomly any page for
replacement
Read-Only Memory
Random access as well as cache memories are
examples of volatile memories
A volatile storage is defined as the one which
loses its contents when power is turned off
Nonvolatile memory storages are those that retain
the stored information if power is turned off
As there is a need for volatile storage there is also
a need for nonvolatile storage
Computer system boot subroutines, microcode
control, video game cartridges are few examples
of computer software that require the use of
nonvolatile storage
ROM can also be used to realize combinational
logic functions
Read-Only Memory Technology
The technology used for implementing ROM chips has evolved
over the years:
 Early implementations of ROMs were called
maskprogrammed ROMs.
 a made-to-order one time ROM is programmed
according to a specific encoding pattern supplied by the
user
 If the user would like to program his/her ROM on site,
then a different type of ROM, called the Programmable
ROM (PROM) should be used
 Although it allows for some added flexibility, yet PROM
is still restricted by the fact that it can only be
programmed once (by the user)
 A third type of ROM, called Erasable PROM (EPROM) is
reprogrammable, i.e., allows stored data to be erased
and new data to be stored
 Flash EPROMs (FEPROMs) have emerged as strong
contenders to EPROMs - FEPROMs are more compact,
faster, and removable as compared to EPROM
Characteristics of Different ROM
Implementation

Lect 03 PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lect 03 PDF

Uploaded by

Copyright:

Available Formats

LECTURE 3: REVIEW OF

MEMORY ORGANIZATION AND

EE326 - EMBEDDED SYSTEMS

USP JULY 2018

• Typical Memory Hierarchy

• Memory Hierarchy Parameters

• Cache, Main, Virtual Memory, ROM

The inverse, however, is not necessarily true. That is,

We call a reference to a missing item a “miss”

A word miss in Mi implies that it is also missing from

The requirement that copies of data items at

If a word is modified in the cache, copies of that

There are two strategies for maintaining the

immediate update of the corresponding data

A miss represents a not found item (the item must

The hit ratio, hi, at Mi is the probability (between 0

The miss ratio at Mi is obviously just 1- hi

We assume h0 = 0 (CPU always accesses M1) and hn =

It represents the probability of successfully

Note that f1 = h1, and f

The access frequencies decrease from low to high

The effective access time of a memory hierarchy can be

 h1t1  (1  h1 )h2t2  L  (1  h1 )(1  h2 )L (1  hn 1 )hntn

* The first several terms in the above expression dominate,

Different organization of the same memory capacity

The available per chip memory capacity can be a

It is possible to use a 1-transistor dynamic cell

Operation of Control Circuitry

 if m  M has been allocated to store

• The mapping returns a physical address (m) if a memory hit

The purpose of memory allocation is to allocate pages

You might also like