Chapter Three

Memory System
 Contents
• Characteristics and hierarchy of memory systems
• Cache memory
• Main memory
• Secondary memory
• Virtual memory
• Memory management
Characteristics and Hierarchy of
Memory Systems
 Characteristics of Memory System
• The memory unit is a essential component in any digital
computer since it is needed for storing programs and data.
1. Location
• Processor
• Internal (main)
• External (secondary)
2. Capacity
• An obvious characteristics of memory is its capacity.
• Capacity of a memory is typically expressed in terms of
bytes (1 byte=8bits) or words.
• Common word length are 8,16 and 32 bits
 Characteristics of Memory System
3. Unit of Transfer
• Unit of transfer is equal to the number of electrical lines into
and out of the memory module.
• For main memory, this is the number of bits read out of or
written into memory at a time.
• For external memory, data are transferred in much larger
units than a word, referred to as blocks
Addressable Units
• In some systems, the addressable unit is the word.
• If we have A bits length of an address, then the number of
addressable unit is N=2A
 Characteristics of Memory System
4. Method of Accessing
• Another distinction among memory types is the method of
accessing units of data.
a) Sequential Access
 Memory is organized into units of data called, records.
 Access must be in specific linear sequence and a shared
read write mechanism is used. (example: Tape units)
b) Direct Access
 Individual blocks have unique address based on physical
location.
 Access is by jumping to vicinity plus sequential search to
reach the final location
 Access time depends on location and previous location
(example: Disk units)
 Characteristics of Memory System
c) Random Access
 Individual addresses identify locations exactly
 Access time is independent of location or previous access
and is constant
(example: main memory/RAM)
d) Associative Memory
 The word is retrieved based on a portion of its contents
rather than its address.
 Access time is independent of location or previous access
(example: cache memory)
 Characteristics of Memory System
5. Performance Parameters
a) Access time(latency)
 For random access: it is the time from the instant that an
address presented to the memory to the time that the
data have been stored or made available for use.
 For non random access: it is the time it takes to position
the read-write mechanism at the desired location.
b) Memory cycle time
 Access time plus any additional time required before a
second access can commence.
 Note: memory cycle time is concerned with the system
bus, not the processor.
 Characteristics of Memory System
c) Transfer rate
 This is the rate at which data can be transferred into or
out of a memory unit.
 For random access:
[Transfer rate = 1/cycle time]
 For non random access memory:
TN= TA+ n/R
Where: TN= the average time to read or write N-bits
TA=average access time
n=number of bits
R=transfer rate, in bits per second (bps)
 Characteristics of Memory System
6. Physical Types
 Semiconductor: RAM
 Magnetic surface: Disk & Tape
 Optical and magneto optical: CD & DVD
7. Physical Characteristics
 Volatile/Non-volatile
 Erasable/Non-erasable
8. Organisation
 Physical arrangement of bits into words
 Hierarchy of Memory
• The design constraints on computers memory can be
summed up by three questions:
1. How much? =Capacity
2. How fast?=access time
3. How expensive? =cost
• Memory systems can be implemented by variety of
technologies, across this spectrum of technologies, the
following relationships holds:
 Faster access time, greater cost per bit
 Greater capacity, smaller cost per bit
 Greater capacity, slower access time
 Hierarchy of Memory
• Generally, the objective behind designing a memory
hierarchy is to have a memory system that performs as
if it consists entirely of the fastest unit and whose cost
is dominated by the cost of the slowest unit
 Hierarchy of Memory
• As one goes down the hierarchy, the following occur
a) Decrease cost per bit
b) Increasing capacity
c) Increasing access time
d) Decrease frequency of access of the memory by the
processor
• The effectiveness of memory hierarchy depends on the
principle of moving information into the fast memory
infrequently and accessing it many times before replacing
it with new information.
• This principle is possible due to a phenomenon called
locality of reference
 Hierarchy of Memory
Locality of reference: with a given period of time, programs
tend to reference a relatively confined area of memory
repeatedly.
There exists two forms of locality:
 Spatial locality
 Temporal locality
• Spatial Locality (locality in space): refers to the
phenomenon that when a given address has been referenced,
it is most likely that addresses near it will be referenced with
in a short period of time.
• Temporal Locality (locality in time): refers to the
phenomenon that once a particular memory item has been
referenced, it is most likely that it will be referenced next.
Example: an instruction in a program loop
Cache Memory
 Cache Memory Principle
• Cache is at the first level of memory hierarchy and used to
keep the information expected to be used more frequently by
the CPU.
• Sits between normal main memory and CPU
• Contains a copy of portion of main memory
• May be located on CPU chip or module
 Cache Memory Principle
• At any given time some active portion of the main memory is
duplicated in the cache.
• Therefore when the processor makes a request for a memory
reference, the request is first sought in the cache.
• If the request corresponds to an element that is currently residing
in the cache, get from cache (fast)==cache hit.
• On the other hand, if the request corresponds to an element that is
not currently in the cache, read required block from main memory
to cache== cache miss.
 A cache hit ratio, hc: probability of finding the requested
element in the cache.
 A cache miss ratio, (1-hc): probability of not finding the
requested element in the cache.
• Cache includes tags to identify which block of main memory is in
each cache slot
Cache/Main Memory Structure
 Cache/Main Memory Structure
• Main memory consists up to 2n addressable words, with each
word having a unique n-bit address.
• For mapping purposes, let us assume this memory consists of
a number of fixed length blocks of k words each.
M=2n/k (number of blocks in main memory)
• The cache consists of m blocks, called lines. Each line
contains k-words, plus a tag of a few bits and other control
bits.
Note:
 The length of a line, not including tag and control bits, is the
line size (m)- (m<<M)
 A tag is usually a portion of the main memory address.
Typical Cache Organization
 Cache Mapping Function
• A request for accessing a memory element is made by the
processor through issuing the address of the requested
element.
• To determine the whereabouts of the requested element
address translation has to be made.
 This is one of the functions performed by the memory
management unit (MMU)
Cache Mapping Function
 Cache Memory Organization
• There are three main different organization techniques
used for cache memory.
• The techniques differ in two main aspects:
1. The criterion used to place, in the cache, an incoming
block from the main memory.
2. The criterion used to replace a cache block by an
incoming block (on cache full)
 Cache Memory Organization
1. Direct Mapping: maps each block of main memory into
only one possible cache line.
 Many to one mapping techniques
 The mapping is expressed as:
i=j modulo m
Where
i=cache line number
j=main memory block number
m=number of lines in the cache
Example
 Cache Memory Organization
• The main advantage of the direct-mapping technique is its
simplicity in determining where to place an incoming main
memory block in the cache.
• The main disadvantage is the inefficient use of the cache.
This is due to, a number of main memory block may compete
for a given cache line even if there exist other empty cache
lines.
• According to direct mapping technique the MMU interprets
the address issued by the processor by dividing the address
into three fields:
 Cache Memory Organization

Generally,
 Address length= (s+w)bits
 Number of addressable units=2s+w words/or bytes
 Block size=line size=2w words/or bytes
 Number of blocks in main memory=2s+w/2w=2s
 Number of lines in the cache=2r=m
 Size of cache=2r+w words/or bytes
 Size of tag=(s-r)bits
Where
• w-->a unique word within a block of main memory
• s-->specify one of 2s blocks of main memory
• The cache logic interprets these s bits as a tag of (s-r) bits and as a
line field of r-bits
• m=2r-->identifies line of cache
 Cache Memory Organization
2. Fully Associative Mapping: an incoming main memory
block can be placed in any available cache block.
 The address issued by the processor need only have two fields
i.e. the Tag and Word fields
Tag-->uniquely identifies the block while residing in the cache
Word-->identifies the element within the block that is requested by
the processor
 The MMU interprets the address issued by the processor by
dividing it into two fields.

 Note: No field in the address corresponds to the line number,

so that the number of lines in the cache is not determined by
the address format.
 Cache Memory Organization
Generally,
 Address length= (s+w)bits
 Number of addressable units=2s+w words/or bytes
 Block size=line size=2w words/or bytes
 Number of blocks in main memory=2s+w/2w=2s
 Number of lines in the cache= undetermined
 Size of tag=s bits
• The advantage of associative mapping is an efficient use of
the cache.
• The principal disadvantage is the complex circuitry required
to examine the tags of all cache lines in parallel.
Example
 Cache Memory Organization
3. Set-Associative Mapping: is a compromise that exhibits the
strength of both the direct and associative approaches while
reducing their disadvantages.
 In this case, the cache consists of a number of sets, each of
which consists of a number of lines.
m=V*K
i=j modulo V K-way set associative mapping
Where
 i-->cache set number to which block j maps
 j-->main memory block number
 m-->number of lines in the cache
 V-->number of sets in the cache
 K-->number of lines in each set
 Cache Memory Organization
• The address issued by the processor is divided into distinct
three fields:

 Set-->used to uniquely identify the specific cache set that

ideally should hold the target block
 Tag-->uniquely identifies the targeted block within the
determined set.
 Word-->identifies the element (word) within the block that
is requested by the processor.
 Cache Memory Organization
Generally,
 Address length= (s+w)bits
 Number of addressable units=2s+w words/or bytes
 Block size=line size=2w words/or bytes
 Number of blocks in main memory=2s+w/2w=2s
 Number of lines in set=K
 Number of sets=V=2d (d is set bits)
 Number of lines in the cache=m=KV
 Size of cache=K*2d+w words/or bytes
 Size of tag=(s-d)bits
Example
 Cache Memory Organization
Replacement Algorithm
• Once the cache has been filled, one of existing blocks must
be replaced before a new block is brought into the cache.
 For direct mapping, there is only one possible line for any
particular block, so we don’t have other choices.
 For associative and set-associative techniques, a
replacement algorithm is needed
Most common replacement algorithms include:
1. Random selection: a randomly selected block
• Random selection of a block for replacement is done based
on the output of the random number generator at the time
of replacement.
 Cache Memory Organization
2. FIFO (First-in-first-out): the block that has been in the
cache the longest.
• It takes the time spent by a block in the cache as a measure
for replacement.
• The block that has been in the cache the longest is selected
for replacement regardless of the recent pattern of access of
the block
3. LRU (Least recently used): the block that has been used
the least while residing in the cache.
• The cache block that has been recently used the least is
selected for replacement.
• This technique is the most effective
 Cache Memory Organization
Write Policy
• Coherence between a cache word and its copy in main
memory should be maintained at all times.
• A number of policies or techniques are used in performing
write operations to the main memory blocks while residing
in the cache.
1. Write -through
 The simplest technique, where all write operations are made
to maintain main memory as well as to the cache, ensuring
that main memory is always valid.
 Every write operation to the cache is repeated to the main
memory at the same time
Disadvantage: generate a substantial memory traffic and may
create bottleneck
 Cache Memory Organization
2. Write-back
 Alternative technique, where only the cache location is
updated during a write operation.
 The location is then marked by a flag so that later when the
word is removed from the cache it is copied into main
memory.
 This technique minimizes memory writes, because updates
are made in the cache.
Main Memory
 Main Memory
• Provides the main storage for a computer
• Relatively large and fast memory is used to store programs
and data during the computer operation
• The principal technology used for the main memory is based
on semiconductor integrated circuits
Integrated circuit RAM chips are available in two possible
operating modes, static and dynamic.
i. Static RAM (SRAM)
 Consists of internal flip-flops that stores the binary
information.
 Stored information remains valid as long as power is
applied to the unit.
 Main Memory
ii. Dynamic RAM (DRAM)
 Employ a capacitor, which stores a binary information in the form of
electric charges.
 Since the stored charge on the capacitors tend to discharge with
time, the capacitors must be periodically recharged by refreshing
frequently.
• Refreshing is done by cycling through the words every few
milliseconds.
 In DRAM capacitors are provided inside the chip by MOS
transistors and these capacitors are much smaller than
flip-flops, and so a capacitors based DRAM can hold
much information in the same area than an SRAM.
 Read Only Memory (ROM)
• Most of the main memory in general-purpose computer is
made up of RAM integrated circuit chips, but a portion of
memory may be constructed ROM chips.
• ROM is needed for storing an initial program called a
bootstrap loader.
 The bootstrap loader is a program whose function is to start the
computer operating system software when power is turned on
 The start up of a computer consists of turning the power on and
starting the execution of an initial program, i.e. when power is turned
on, the hardware of the computer sets the program counter (PC) to
the first address of the bootstrap loader.
Note: the bootstrap program loads a portion of the operating
system from disk to main memory and control is then
transferred to the operating system, which prepares the
computer for general use.
Secondary Memory
 Magnetic Disk
• A magnetic disk is a circular plate constructed of metal or
plastic coated with magnetized material.
• Often both sides of the disk are used and several disks may
be stacked on one spindle with read/write heads available on
each surface.
• Bits are stored in the magnetized surface in spots a long the
concentric circles called tracks.
• The tracks are commonly divided into sections called
sectors.
• A sector is the smallest addressable unit in a disk
• There are thousands of tracks per surface and hundreds of
sectors per track
Disk Layout
 Magnetic Disk…
• The bottleneck of a disk access is moving the read/write
arm. So it makes sense to store a file in tracks that are
below/above each other in different surfaces, rather than in
several tracks in the same surface.
Cylinder: the set of tracks on a
disk that are directly above/below
each other.
 All the information on a
cylinder can be accessed
without moving the read/write
head.
Components of a Disk Drive
 Magnetic Disk…
• Number of cylinders=number of tracks in a surface
• Track capacity=number of sectors per track*Bytes per sector
• Cylinder capacity=number of surfaces*track capacity
• Disk capacity=number of cylinder*cylinder capacity
 Magnetic Disk…
• A disk system is addressed by address bits that specify:
 The disk number
 The disk surface
 The sector number
 The track within the sector
Operation
• After the read/write heads are positioned in specified track,
the system has to wait until the rotating disk reaches the
specified sector under the read/write head.
• Note: Disks may have multiple heads and simultaneous
transfer of bits from several tracks at the same time.
 Disk Performance Parameters
• The actual detail of disk I/O operation depend on the:
Computer system
Operating system
Nature of I/O channel
Disk controller hardware
1. Seek time: the time it takes to position the head at the track.
2. Rotational delay (latency): the time it takes for the
beginning of the sector to reach the head.
3. Access time: the sum of seek time, if any, and the rotational
delay i.e. the time it takes to get into position to read or write
4. Transfer time: the time required to transfer a data from a
sector in specified track.
 Disk Performance Parameters
Transfer time:
T=b/rN
Where: T-->transfer time
b-->number of bytes to be transferred
N-->number of byte on a track
r-->rotation speed, rps
 Total average access time can be expressed as:
Ta=Ts+1/2r+b/rN
Where: Ts-->average seek time
Example
Virtual Memory
 Virtual Memory
• In a memory hierarchy system, program and data are first
stored in secondary memory (hard disk).
• Portion of a program and/or data are brought into main
memory as they are needed by the processor.
• It is usually apparent that the available physical memory
space will not be enough to host all the parts of a given active
program.
• In computing, virtual memory is a memory management
technique that is implemented using both hardware and
software.
• It maps memory addresses used by a program, called virtual
addresses, into physical addresses in computer memory.
 Virtual Memory
• The most relevant principle is that of keeping active
segments in the high-speed main memory and moving in
active segments back to the hard disk.
• Movement of data between the disk and the main memory
takes the form of pages.
A page: is a collection of memory words, which can be moved
from the disk to the main memory when the processor requests
accessing a word on that page
 A typical size of a page in modern computers ranges from 2k
to 16k bytes.
 A page fault occurs when the page containing the word
required by the processor doesn’t exist in the main memory
and has to be brought from the disk.
 Address space and Memory space
• An address used by a programmer is called a virtual address,
and the set of such addresses is called the address space
• An address in main memory is called a physical address. The
set of such locations is called the memory space.
• Note: In a computer with virtual memory, address space is
larger than memory space.
• The main memory management unit (MMU) is responsible
for the translation of virtual addresses to their corresponding
physical addresses.
Page Table: contains information about main memory location
and the corresponding virtual pages, which is stored in main
memory.
 Address mapping using pages
• Consider a computer with an address space of 8k and
memory space of 4k. If we split each into groups of 1k words
we obtain eight pages and four blocks. At any given time, up
to four pages of address space may reside in main memory in
any one of the four blocks.

Virtual address:

Page number Line number

 Address mapping using pages
Line number address bits=log 2(210)=10bits-->line number
within the page
Page number address bits=Address space bits-Line number
address bits
= (13-10)bits =3 bits--> specify one of 8-pages
Note: line address in address space and memory space is the
same
Address mapping using pages
 Address mapping using pages
• The figure shows pages 1,2,5 and 6 are now available in main
memory in blocks 3,0,1 and 2, respectively.
• A presence bit in each location indicates whether the page
has been transferred from auxiliary (secondary) memory into
main memory.
 A 0 in the presence bit indicates that this page is not available
in main memory
Memory Management
 Memory Management Hardware
• A memory management system is a collection of hardware
and software procedures for managing the various programs
residing in memory.
• The memory management software is part of an overall
operating system available in many computers.
 Memory Management Hardware
The basic components of memory management units are:
1. A facility for dynamic storage relocation that maps logical
memory references into physical memory addresses.
2. A provision for sharing common programs stored in
memory by different users.
3. Protection of information against un authorized access
between users and preventing users from changing
operating system function.
 Segmented-Page Mapping
• A segment is a block of continuous locations of varying size.
• Segments are used by the operating system to relocate
complete program in the main and the disk memory.
• The address generated by a segmented program is called a
logical address.
 Logical address is partitioned into three fields:

Segment Page Word

 Segment: specifies segment number
 Page: specifies the page within the segment
 Word: gives the specific word within the page
Logical to physical address mapping
 Segmented-Page Mapping
• Segment table and page table may be stored in two separate
small memories or in main memory.
• In either case, a memory reference from the CPU will require
three access to memory:
1. From segment table
2. From page table
3. From main memory
• This would slow the system significantly when compared to a
conventional system that requires only one reference to
memory
• To avoid this speed penalty, a fast associative memory is used
to hold the most recently referenced table entries.
 This type of memory is called a translation look-aside buffer,
TLB
Any Q???