Advanced Computer Architecture

CS-522

MS – Computer Science

Credit Hours : 3-0

Dr. Shahid Latif (Associate Professor)

Department of Computer Science & IT

1
Sarhad University of Science and Information Technology, Peshawar
Course Details
Course title/code: Adv. Computer Architecture/CS-522

Lecture: 04
Topic: Cache Memory

Program: MS – Computer Science (Sem: 1st, 2nd, 3rd & 4th)

Department of Computer Science & IT

Sarhad University of Science and Information Technology, Peshawar 2
Lecture Outlines
• Computer Memory System
Overview
• Characteristic of Memory System
• Location, Capacity, Unit of
transfer, Access method,
Performance, Physical type,
Physical characteristics,
Organization etc.
• The Memory Hierarchy
• Cache Memory Principles
• What is Cache?
• Cache/Main memory
structure
• Cache Operation
• Cache Memory
• Cache and Main Memory
Structure
• Elements of Cache Design
• Cache Addresses: Logical
and Physical addresses
• Cache Size
• Mapping Function
• Direct Mapping
• Associative Mapping
• Set-Associative
Mapping
3
Characteristics of Computer Mem.
• Location
• Capacity
• Unit of transfer
• Access method
• Performance
• Physical type
• Physical characteristics
• Organisation
Location of computer memories
• Internal (main memory)
– Processor (local memory)
• Registers PC, ACC, MQ, MAR, MBR, IR IBR etc.
– Main memory (RAM), Cache

• External (secondary)
– Optical disks, magnetic disks and tapes
Capacity of computer memories
• Capacity of Internal Memory
– Typically expressed in bytes (8-bits) or words
– Word size: the natural unit of organisation
– i.e. 8, 16 or 32 bits
• Capacity of External Memory
– Typically expressed in bytes (8-bits)
– KB, MB, GB or even TB etc.
Unit of Transfer
• Internal memory
– Usually governed by data bus width (no: of data lines into and out of
memory module)
– May be equal to the word length, but is often larger, such as 64, 128, or
256 bits
– Word = Addressable unit (= word or byte)
– Smallest location which can be uniquely addressed

• External memory
– Usually a block, usually much larger than a word
Access Methods
• Sequential
– Access must be in a specific linear sequence
– Start at the beginning and read through in order
– Access time depends on location of data and previous location
• Thus highly variable
– e.g. Tape
• Direct
– Individual blocks have unique address based on physical locations
– Access is by jumping to vicinity (surrounding area) plus sequential
searching/counting/waiting
– Access time depends on location and previous location
• Thus variable
– e.g. Disk
Access Methods
• Random
– Individual addresses identify memory locations exactly
– Access time is independent of location or previous access
– e.g. Main memory (RAM) and some Cache

• Associative
– Data is located by a comparison with contents of a portion of the store
– i.e. word is retrieved based on a portion of its contents rather than its address
– Access time is independent of location or previous access
– e.g. Cache (Random Access type)
Performance
• Access time (latency)
– Time between presenting the address to memory and getting the valid
data
– Time it takes to perform a read or write operation
• Memory Cycle time
– Time may be required for the memory to “recover” before next access
– Memory Cycle time = access time + additional time (recovery)
• Transfer Rate
– Rate at which data can be moved (into or out of memory)

Recovery: regeneration of data after read-destructive

Physical Types
• Semiconductor
– RAM
• Magnetic
– Disk & Tape
• Optical
– CD & DVD
• Others (such as magneto-optical)
– Bubble
• A thin film of a magnetic material to hold small magnetized areas, known
as bubbles or domains, each storing one bit of data
Physical Characteristics
• Volatile/non-volatile
– Volatile: information is lost when electrical power is switched off
– Non-Volatile: no electrical power is needed to retain information
• Semiconductor memories may/may not be volatile
• Magnetic surface memories are non-volatile
• Erasable/non-erasable
– Non-erasable: cannot be altered, except by destroying the storage
unit
• such as read-only memory (ROM)
– A practical non-erasable memory must also be nonvolatile

• Power consumption
Organisation
• Physical arrangement of bits to form words
• Not always obvious
– e.g. Interleaved
• (to insert something alternately and regularly between the parts of…)
Memory Hierarchy
• Registers
– In CPU
• Internal or Main memory
– May include one or more levels of cache
– “RAM”
• External memory
– Backing store
– HD, FD, CD, DVD, USB, Tapes etc.
Memory design constraints
• How much?
– Capacity
• How fast?
– Access time
• How expensive?
– Cost per bit

• Relationships among the three (capacity, access time, cost)

holds:

– Faster access time, greater cost per bit

– Greater capacity, smaller cost per bit
– Greater capacity, slower access time
 Memory Hierarchy - Diagram

As one goes down the hierarchy

a. Decreasing cost per bit

b. Increasing capacity
c. Increasing access time
d. Decreasing frequency of access
of memory by processor
So you want fast?
• Smaller, more expensive but faster memory (i.e. 1000 words, access
time=0.01µs) is supplemented (complemented) by larger, cheaper but
slower memories (i.e. 100,000 words, access time= 0.1µs)

• It is possible to build a computer which uses only static RAM

(see later)
– This would be very fast
– This would need no cache
– This would cost a very large amount
Cache Memory Principles

18
Cache
• Small amount of fast memory
• Sits between normal main memory (RAM) and CPU
• May be located on CPU chip or module

• Contains a copy of portions of main memory

– When the processor attempts to read a word of memory,
– a check is made to determine if the word is in the cache
– If so, the word is delivered to the processor
– If not, a block of main memory (some fixed number of words) is read
into the cache and then the word is delivered to the processor
20
 Cache/Main Memory Structure

Tag = portion of main memory address

C (no: of lines) << M (no: of blocks)

Cache operation – overview
• CPU requests contents of memory location
• Check cache for this data
• If present, get from cache (fast)
• If not present, read required block from main memory to cache
• Then deliver from cache to CPU
• Cache includes tags to identify which block of main memory is
in each cache slot
• C (no: of lines)<<M (no: of blocks),
– a single line of cache can not be uniquely and permanently dedicated to the one
block
– So tags are used to identify which block is stored
Cache Read Operation
Flowchart
Typical Cache Organization
Elements of Cache Design

25
Cache Addresses
• A logical cache (virtual cache)
– stores data using virtual addresses
– processor accesses the cache directly,
• without going through the MMU (Memory Management Unit)
– Advantage: cache access speed is faster than for a physical cache
• as cache can respond before the MMU performs an address translation
– Disadvantage: virtual memory supply each application with the same
virtual memory address space
• each application sees a virtual memory that starts at address 0
• each line of cache need to identify which virtual address space this address
refers to
• A physical cache
– stores data using main memory physical addresses
– processor accesses the cache going through the MMU
26
27
Cache Size
• Cache (size) would be liked small enough so that,

– Cost
• Cost of cache = cost of main memory (approximately)
• More cache is expensive

– Speed
• More cache is faster (up to a point)
• Large cache means large number of gates (for addressing the cache)
– Tend to be slightly slower than smaller ones

– Chip and board area

28
• Available chip area also limits cache size
Comparison of Cache Sizes

29
Mapping Function
• As fewer cache lines than main memory blocks
– An algorithm is needed for mapping main memory blocks into cache
lines
• determining which main memory block currently occupies a cache line
– Three techniques are used
• Direct
• Associative
• Set-associative

• For all three cases: example include

– Cache of 64kB
– Cache line/block of 4 bytes (each line=32-bits)
• i.e. cache is 16k (214) lines of 4 bytes each
– 16MB main memory
• 24 bit address i.e. (224=16M) 30
• 4M blocks of 4 Bytes each
Direct Mapping
• Each block of main memory maps to only one cache line
– i.e. if a block is in cache, it must be in one specific place

• Main memory address is in two parts (3-fields)

– Least Significant w bits (2 bits) identify unique word within a block
– Most Significant s (22 bits) bits specify one memory block
– The MSBs are split into
• a cache line field r and
• a tag of s-r (most significant)

Tag s-r Line or Slot r Word w

8 14 2
31
Direct Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2s + w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s+w/2w = 2s
• Number of lines in cache = m = 2r
• Size of cache = 2r + w
• Size of tag = (s – r) bits

32
 Direct Mapping Cache Line Table

Cache line Main Memory blocks assigned

0 0, 100, 200, 300,…,

1 1,101, 201, 301,…,
--- -------------------
33
 Direct Mapping Address Structure

Tag s-r Line or Slot r Word w

8 14 2

• 24 bit address
• 2 bit word identifier (4 byte block)
• 22 bit block identifier
– 8 bit tag (=22-14)
– 14 bit slot or line
• No two blocks in the same line have the same Tag field
• Check contents of cache by finding line and checking Tag
Direct Mapping Cache Organization

35
Direct Mapping
Example

36
Direct Mapping pros & cons
• Simple
• Inexpensive to implement
• Disadvantage: Fixed location for given block
– If a program accesses 2 blocks that map to the same line repeatedly,
cache misses are very high

37
Quiz-01
A short quiz is scheduled for the class of the upcoming week

The quiz comprises:

2-3 Questions
Duration: 20-30 mints
Worth: 10 Marks
Wrapping Lectures- 1-4
 Thank you

Dr. Shahid Latif (Associate Professor)

Department of Computer Science & IT
Sarhad University of Science and Information Technology, Peshawar

39