CO6:

Analyze the levels of computer memory and its

operation (registers, main memory, cache memory, etc.).
Cache Memory
Cache and Main Memory
 Small amount of fast memory
 Sits between normal main memory and CPU
 May be located on CPU chip or off chip
Cache Memory
 Cache memory is built out of static memory.

 Cache memory is faster to access than Main memory

(dynamic memory).

 Cache mechanism is effective because of locality of

reference.

 In computer programs, many instructions are executed

repeatedly.
Locality of Reference
 In computer programs, many instructions are executed
repeatedly. (LOOPs)

 Many instructions in localized areas are executed

repeatedly and remainder of programs is accessed relatively
infrequently.

 This is the locality of reference:

 Recently executed instructions are likely to be executed very soon.
 Instructions in close proximity to a recently executed instruction are
also likely to be executed soon.

 Then, Place the active segment of the program in a fast

cache memory (Speed is higher).
Cache memory

 Whenever an item is needed, it should be brought into

Cache where it is hopefully needed again. (Also bring the
addresses that are around it).
Cache
 A block = Cache line = a number of memory
locations
 Cache memory has a small number of blocks
 Main Memory has a large number of blocks

Cache Main
Memory

Small number of blocks

Large number of blocks

Operation of Cache

 CPU dos not know about the cache

 CPU issues a Read/Write requests and addresses the
main memory.
 The cache control circuit checks whether the words
exist in the cache

 If YES, do Read/Write (Read/Write Hit has occurred)

 If NO (Read/Write Miss has occurred)

Read/ Write Hit (The addressed word is in Cache)
 In Read Hit Operation
 Main memory is not involved

 In Write Hit Operation

 There are two possible protocols

➢ If Write Through Protocol

Cache and Main Memory is updated simultaneously.

➢ If Write Back protocol

▪ Update Cache location and mark it as Updated with a flag
(dirty bit).
▪ Main Memory is updated when the block is removed from
Cache.
Cache Read Operation - Flowchart

RA= Read
Address Read Miss

Read Hit

Using Replacement Algorithm

Read/ Write Miss (The addressed word is not in
Cache)

 In Read Miss Operation

 Bring the block to Cache
 Read the proper word

 In Write Miss Operation

➢ If Write-Through Protocol
Data is written to Main memory.

➢ If Write Back protocol

▪ Block is brought to Cache
▪ Update the Cache location
Cache Operation

 Mapping Function: Correspondence between the

main memory blocks and those in Cache.
 Direct Mapping
 Associative Mapping
 Set-Associative Mapping

 Replacement Algorithm: Cache control circuit must

decide which block should be removed to create
space for the new block that contain the referenced
word.
 Mapping Functions
Block 0
 Direct Mapping (Example)
Tag = 0
 Assume
 Main Memory=64k word
Block 127
(Add. Bus=16 bits)
Block128
 Cache =2k word
Tag
 Block=16 locations (word) Tag = 1
Block 0

Block 255

Tag Block 256

MM= 64k
4096 block
Block 127

Cache=2k
128 block

Tag = 31
There is 4096/128 = 32 different blocks in MM
could reside in the same block location in Cache
Block 4095
 Direct Mapping
 In Direct Mapping, each block in the memory could reside in
only one block location in Cache.

 Block j in Main Memory maps to block j mod 128 in Cache.

 Blocks 0, 128, 256, …. is stored in Cache block 0
 Blocks 1, 129, 257, …. is stored in Cache block 1
 And so on.

 Disadvantage: Contention may arise for certain Cache

position even when Cache is not full.

 Replacement algorithm is trivial: new block replaces the old

block. Positions are predetermined.
 Direct Mapping
There are 32 different blocks in MM could reside in the same position in
Cache.

Tag = 4096/128 = 32

Tag is associated with each block in Cache to differentiate between 32

different block.

Tag needs 5 bits

Block needs 7 bits
Word need 4 bits

Tag Block Word

Memory Address 5 7 4
Format
 Direct Mapping
Tag Block Word

Memory Address 5 7 4

The higher order 5 bits in Memory address are stored in a 5 bits

tag register associated with its location in Cache.

How Cache determine whether it has referenced data or not.

-First, 7 bits block determine which block in Cache
Then, 5 bits tag is compared to decide if the required word in
in Cache or not.

Direct Mapping is easy but not flexible.

Direct Mapping

Example: Assume A cache with direct mapping and

MM=128 k word
Cache Memory= 1 k word
Block size= 8 words

Determine the format of the address sent to MM

Direct Mapping
Example: Assume that the
MM=128 k word
Cache Memory= 1 k word
Block size= 8 words
Tag Block Word

Memory Address 7 7 3

The higher order 7 bits in Memory address are stored in a 7 bits

tag associated with each block.
How Cache determine whether it has referenced data or not.
-First, 7 bits block determine which block in Cache
Then, 7 bits tag is compared to decide if the required word in
in Cache or not.
 Associative Mapping
 Replace block by new one if Cache is full.

 Any MM block could reside in any place in Cache.

 In any Cache position, there are 4096 blocks in MM could

reside.
 It needs 12 bit tag to identify which block reside in Cache.

 How Cache determine whether it has referenced data or not.

 CPU sends an address, the tag bits of each block in Cache are
compared with the tag of the address to check if the desired
block is present.
 If block is not present, then a replacement algorithm.
Associative Mapping
Tag Word

Memory Address 12 4
Format

 More flexible
 But Cost to search 128 tag is high.

Replacement Algorithm:
 When all positions are full, Cache controllers decide which
block to remove.
 Replace Least Recently Used (LRU) block.
 Set - Associative Mapping
 Cache blocks are grouped into sets. (For example a set could
contain 2 blocks - it is called 2-way set associative).

 A MM block could reside in any block within a specific set in

Cache.

 Replace block by new one if Cache Set is full.

 In any Cache position, there are 2 x 32=64 blocks in MM

could reside.
Set-Associative Mapping
Set 0 Block 0
Set 1 Block 1
.
Set 63
Set 0
.
Block 127
Set 63
Block128

Tag
Block 0
Set 0 Block 1

Block 255

Block 256
MM= 64k
Block 126
Set 63 4096 block
Block 127
Tag

Cache=2k
128 block

There is 4096/64 = 64 different blocks in MM

could reside in the same block location in Cache
Block 4095
 Set - Associative Mapping
 It needs 6 bit tag to identify which block reside in Cache
(i.e.64 blocks could reside in a block position).
 It needs 6 bit Set field to determine which set in the cache
might contain the desired block.
 (the cache has 64 sets.)
 It needs 4 bit Word to access the word

Tag Set Word

Memory Address 6 6 4
Format
 Set - Associative Mapping
How Cache determine whether it has referenced data or not.
-First, 6 bits set determine which set in Cache.
Then, 6 bits tag is compared to all tags in the set to decide if the required
word is in Cache or not.

 Set associative mapping is in the middle ground between direct

mapping and associative mapping.

 More flexible than direct mapping

 But Cost of search is more than direct mapping.

Replacement Algorithm:
 When all positions in set are full, Cache controllers decide
which block to remove from the set.
 Replace Least Recently Used (LRU) block.
Set - Associative Mapping
Replacement Algorithm
- Use Least Recently Used (LRU) block to be replaced.
- Assume set associative mapping with each set contains 4
blocks.
- To track the accessing of each block, we need two bit counter
associated with each block.
 LRU Algorithm
- To track the accessing of each block, we need two bit counter
associated with each block.
- When a hit occurs:
- Set counter of block to 0
- Increment lower counters by 1
- All others remain the same
- When a miss occurs:
- (Cache is not full)
- Set new counter block to 0

- Increment all other counters by 1

- (Cache is full)
- Remove block with counter=3

- Put in place new block and set its counter to 0

- The other three counters are incremented by 1

Reducing Cache Misses
- Increase Cache size (but increase cost)
- Increase block size ( but increase miss penalty)
- Higher associativity (but increase hit time)
Example:
 Assume
 A cache with 128 blocks of data,
 A main memory with 256 blocks of data,
 Each block consists of 8 words.

Show the format of the main memory address if the cache

uses:

 - direct mapping
 - associative mapping
 - set-associative mapping with 2 blocks per set
Example:
 Assume
 A cache with 128 blocks of data,
 A main memory with 256 blocks of data,
 Each block consists of 8 words. Show the format of the main memory
address if the cache uses
Tag Block Word
 - direct mapping
1 7 3

Tag Word
 - associative mapping
8 3

 set-associative mapping with 2 blocks per set Tag Set Word

2 6 3
 Other Replacement Algortihms

 Least Recently Used

 First In First Out
 Least Frequently Used
 Random (No Cost)