Lecture23

CacheCoherenceProtocols

The3CsofCacheMisses:Revisited
Compulsory
p
y
Capacity
p
y
Conflict
Conflict
The4th C:Coherence

EE/CS520 Comp.Archi.

11/22/2012

CacheCoherence:BasicIdea
Cachesstoretheirowncopiesofthedata
Thosecopiescaneasilygetinconsistent
Those copies can easily get inconsistent

EE/CS520 Comp.Archi.

11/22/2012

HowtoImplementCacheCoherence?
Sharedcaches
Slightlyenforcescoherence
Notscalable(L1cachequicklybecomesabottleneck)

Snooping
Needsabroadcastmedium(likeabus)toenforcecoherence
N d b d t
di
(lik b ) t
f
h
Eachcachethathasablocktracksitssharingstateonitsown
Allcachecontrollerssnoopthemediumtodetermineiftheyhave
p
y

acopyofrequestedblock
Directory
Ablockhasjustoneplacewhereitsfullsharingstateiskept
A bl k h j t
l
h
it f ll h i
t t i k t

(directory)
MorescalablethanSnoopingbasedapproachbuthasmore
overhead
4

EE/CS520 Comp.Archi.

11/22/2012

Snooping
Typicallyusedforbusbased(SMP)multiprocessors
Serializationonthebus
Serialization on the bus coherenceproperty3
coherence property 3
Twoflavors
Writeupdate (writebroadcast)
Awritetoshareddataisbroadcastedtoupdateallcopies
A write to shared data is broadcasted to update all copies
Allsubsequentreadswillreturnthenewvalue(property2)
Allseethewritesintheorderofbroadcasts
Onebus oneorderseenbyall(property3)
y (p p y )
Requiresmorebandwidth
Writeinvalidate
Writetoshareddataforcesinvalidationofallothercachedcopies
p
Subsequentreadsresultincachemissandfetchnewvaluefrom
memory(property2)
Writesorderedbyinvalidationsonthebus(property3)
Requireslessbandwidth
Requires less bandwidth
5

EE/CS520 Comp.Archi.

11/22/2012

Snooping:Efficacy

WriteThrough
WriteBack

EE/CS520 Comp.Archi.

WriteUpdate
(broadcast)

WriteInvalidate

OKChoice,Needsone
OK
Ch i N d
busbutmoreBW
NotGood,Needsmore
,
g
y
BW,ifbroadcastingwhy
notwritethrough

NotGood,NeedsTwo
N
tG d N d T
Busses
BetterChoice,Needs
l
lessBWandoneBus
BW d
B
11/22/2012

SnoopingProtocol:Implementation

EE/CS520 Comp.Archi.

11/22/2012

StatusBitsinSMPCaches
Validbit
Normalcachetagscanbeusedforsnooping
Normal cache tags can be used for snooping
Validbitcanbeusedtoinvalidateablockwhenwritten
Dirtybit
Checkedtoseeifablockislocallyupdated,incaseofareadrequest
Sharedbit
Usedincaseofwritestoasharedmemory
Used in case of writes to a shared memory
Helpsdecidingwhetheraninvalidatemustbegeneratedornot
Ifset blockissharedandwritingtoitmustgenerateaninvalidate
Thegeneratedinvalidationmarkstheblockasexclusive
Ifnotset blockisnotshared,noneedtosendinvalidateonwrites
Theprocessoriscalledownerofthecacheblock
8

EE/CS520 Comp.Archi.

11/22/2012

SnoopingProtocolImplementation
ImplementedasaFiniteStateMachine(FSM)
Thecontrollerrespondstorequestsfromprocessor&bus
Changesthestateoftheblock
Also,usesthebustoaccessdataortoinvalidateit
Also uses the bus to access data or to invalidate it

3statesofadatablockBinFSM
Modified:indicatesthatBhasbeenupdatedincache
p
ImpliesthatBisexclusive

Shared:indicatesthatBispotentiallyshared
Invalid:Bisnotvalid,mustgotomemoryifBisneeded
Invalid B is not valid must go to memory if B is needed

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(0)

AlsocalledModified

FSMportionforCPUrequests
FSM
ti f CPU
t
(MyCPUdoingsomething)
10

EE/CS520 Comp.Archi.

FSMportionforbusrequests
FSM
ti f b
t
(otherCPUsdoingsomething)
11/22/2012

MSISnoopyProtocol(1)

Myprocessorsread/writerequest
Myaction
Someoneelsesrequest
Myaction
11

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(2)
BlockState=Invalid
MyCPUhasareadmiss
M CPU h
d i
Datablockisstale

Willbetreatedasanormalmiss
Placeareadmissonbus
Changeitsstatustoshared
Asnowwe
As now well
llhaveupdatedcopy
have updated copy

12

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(3)
BlockState=Invalid
MyCPUhasawritemiss
M CPU h
i
i
Willbetreatedasanormalmiss
Placeawritemissonbus
Changeitsstatustoexclusive
Wellupdatethisblock

13

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(4)
BlockState=Exclusive
MyCPUhasawritemiss
M CPU h
i
i
Willbetreatedasareplacement
Causedduetoaddressconflict

Writebackthedatablock
Placeawritemissonbus

14

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(5)
BlockState=Exclusive
MyCPUhasawritehit/readhit
M CPU h
i hi / d hi
Willbetreatedasnormalhit
Writedatainthecache

15

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(6)
BlockState=Exclusive
MyCPUhasareadmiss
M CPU h
d i
Willbetreatedasareplacement
Causedduetoaddressconflict

Writebacktheblock
Placeareadmissonbus
Changedstatustoshared
Changed status to shared
Aswellnowhavethecommoncopy

16

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(7)
BlockState=Exclusive
SomeotherCPUhasareadmiss
S
h CPU h
d i
Causescachecoherence
Placecachedblockonbus
Changedstatustoshared

17

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(8)
BlockState=Exclusive
SomeotherCPUhasawritemiss
S
h CPU h
i
i
Causescachecoherence
Writebackthecacheblock
Changedstatustoinvalid
AsotherCPUwillupdateit

18

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(9)
BlockState=Shared
MyCPUhasareadhit
M CPU h
d hi
Willbetreatedasanormalhit
MyCPUwillreaddataincache
y

19

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(10)
BlockState=Shared
MyCPUhasareadmiss
M CPU h
d i
Willbetreatedasareplacement
Thereisanaddressconflictand

theblockwillbefetchedfrom
mainmemorythroughbus
Placeareadmissonbus

20

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(11)
BlockState=Shared
MyCPUhasawritehit
M CPU h
i hi
Causescachecoherence
Placeaninvalidateonbus
Changetheblockstateto

exclusiveormodified

21

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(12)
BlockState=Shared
MyCPUhasawritemiss
M CPU h
i
i
Willbetreatedasareplacement
Thereisanaddressconflict

Placeawritemissonbus
Causescachecoherence
ChangethestatetoExclusiveas
Change the state to Exclusive as

wellupdatethisblock

22

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(13)
BlockState=Shared
SomeotherCPUhasareadmiss
S
h CPU h
d i
Noactionrequired
Letmemoryservethatreadmiss
y

23

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(14)
BlockState=Shared
SomeotherCPUhasinvalidated
S
h CPU h i
lid d

thisblock
Causescachecoherence
Changeitsstatustoinvalid

24

EE/CS520 Comp.Archi.

11/22/2012

MSISnoopyProtocol(15)
BlockState=Shared
SomeotherCPUhasawritemiss
S
h CPU h
i
i

forthisblock
Causescachecoherence
Changeitsstatustoinvalid

25

EE/CS520 Comp.Archi.

11/22/2012

Review:Snooping
IfablocksstateisModified/Exclusive,statethe

valueofthefollowing:
ValidBit:

1
DirtyBit:
Dirty Bit:
1
SharedBit: 0

26

EE/CS520 Comp.Archi.

11/22/2012

LimitationstoSMPsandSnooping
SMPsapproachisnotscalable
Asno.ofprocessorsgrows,memorydemandgrows

Inbusbasedinterconnection,bushastosupportboth

coherencetrafficaswellasthenormalmemorytraffic
Similarly,ifonemainmemory,itmustaccommodateall

processorsrequests
processors
requests
Solution???
27

EE/CS520 Comp.Archi.

11/22/2012

LimitationstoSMPsandSnooping
Usemultiplebussesorcrossbarconnections
Usemultiplemainmemories
p

28

EE/CS520 Comp.Archi.

11/22/2012

ProblemswithSnoopyProtocol
Tremendousburdenofcommunicationtraffic
Oneachcachemiss,communicateittoallcaches
Notscalable
Inexpensive (advantage,nocentralizeddatastructure)
(advantage no centralized data structure)

Needtoreducethebroadcastingtraffic
Need to reduce the broadcasting traffic
Directory
Directorybased
basedprotocolwasdevisedforthepurpose
protocol was devised for the purpose

29

EE/CS520 Comp.Archi.

11/22/2012

TypesofCoherenceMisses

30

EE/CS520 Comp.Archi.

11/22/2012

TypesofCoherenceCacheMisses
TrueSharingMiss
Anactualneedfordatacommunicationthrough

coherencemechanism
BlockXissharedbetweenCacheAandB
Block X is shared between Cache A and B
WordYinBlockXismodifiedbyA,BlockXisinvalidatedinB
WordYisreadbyB,readmissandupdateddataisprovidedbyA

FalseSharingMiss
Notrueneedfordatacommunication
Block
BlockXissharedbetweenCacheAandB
X is shared between Cache A and B
WordYinBlockXismodifiedbyA,BlockXisinvalidatedinB
WordZinBlockXisreadbyB,readmissandsameoldvalueisre
fetchedbyB
31

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol

32

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol
Adirectorykeepstrackofthestateofeveryblockthatcanbecached
Foreachblock,directorystoresthefollowinginfo:
Whichprocessorshavecachedit!
Whetheritisdirtyornot!,etc.
Whether it is dirty or not!, etc.
Simplestimplementation
Foreachmemoryblockassociateanentry
Amountofinfotobestored:no.blocksxno.processors
Feasibleforasystemofaround200processors
Beyondthatreduceeitherofthetwonumbers
Beyond that reduce either of the two numbers
Toavoidbottleneck,directoryisdistributedalongwithmemory

33

EE/CS520 Comp.Archi.

11/22/2012

StatusBitsinDirectorybasedProtocol
LikeSnoopyprotocol,implementedasanFSM
TwoFSMs:
Individualcache
Directory

Same3statesofadatablockBinindividualcacheFSM
Shared
Modified
Invalid

3statesofadatablockBindirectory
Uncached:NoprocessorhasB
Modified:ExactlyoneprocessorhasthemodifiedcopyofB.
y
p
py
Shared:Morethan1processorshasB
34

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:CacheEnd

SimilarstatesandtransitionsforCacheFSMasinsnoopyprotocol
35

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd

36

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(1)
ecto y based otoco
ecto y d ( )
BlockState=Uncached
Readmiss
R d i
Therequestingprocessoris

providedblockfrommemory
StatusischangedtoShared
RequestorisaddedtoSharers{}

37

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(2)
ecto y based otoco
ecto y d ( )
BlockState=Uncached
Writemiss
Wi
i
Therequestingprocessorissentthe

value
StatusischangedtoExclusive
Requestorwillhavetheupdatedcopy

Requestorisonlymemberin
q
y

Sharers{}

38

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(3)
ecto y based otoco
ecto y d (3)
BlockState=Shared
Readmiss
R d i
Therequestingprocessorissent

thevalue
StatusremainsShared
RequestorisaddedtoSharers{}

39

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(4)
ecto y based otoco
ecto y d ( )
BlockState=Shared
Writemiss
Wi
i
Therequestingprocessorissent

thevalue
AllprocessorsinSharers{}aresent

invalidatemessages
Therequestorwillupdatetheblock

Thestateinmadeexclusive
Therequestorisonlymemberin

Sharers{}
{}

40

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(5)
ecto y based otoco
ecto y d (5)
BlockState=Exclusive
Readmiss
R d i
Therequestissenttoowner

processortoprovideupdatedvalue
StatusischangedtoShared
Ownersendsthedatatodirectory
Updateddataiswrittentomemory
Updated data is written to memory
Therequestingprocessorissent

thevalue
RequestorisaddedtoSharers{}
R
t i dd d t Sh
{}

41

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(6)
ecto y based otoco
ecto y d (6)
BlockState=Exclusive
Writemiss
Wi
i
Theblockhasanewowner
Oldownerissentamessageto
g

invalidate
Oldownersendsthevalueto

directory
y
Therequestingprocessorissent

thevalue,
Therequestorisonlymemberin
The requestor is only member in

Sharers{}
StatusremainsExclusive

42

EE/CS520 Comp.Archi.

11/22/2012

DirectorybasedProtocol:DirectoryEnd(7)
ecto y based otoco
ecto y d ( )
BlockState=Exclusive
Writeback
Wi b k
Theownerhastoreplacetheblock
So,mustwriteitback

Theownersendsthedatato

directory(mainmem)
Updateddataiswrittentomemory
p
y
Since,theownerwastheonly

sharer,thereisnomoresharer
Sharers{}isempty
Sharers {} is empty

StatusischangedtoUncached

43

EE/CS520 Comp.Archi.

11/22/2012

Review:Directory
IfablocksstateisSharedindirectory,statethe

valueofthefollowing:
DirtyBit:

44

EE/CS520 Comp.Archi.

11/22/2012

Quiz6:Thursday,29th Nov.2012
Syllabus:Lecture1923(Caches,MPC,CacheCoherence)
BookChapters:
4thEdition:AppendixC:C.1toC.3,4.1,4.2,4.4,5.1,5.2
5thEdition:AppendixB:B.1toB.3,5.1,5.2,5.4,2.1,2.2

Therecanbenumericalproblems,shortquestionsand/or

objectives...
objectives

45

EE/CS520 Comp.Archi.

11/22/2012