How Caches Work

Y
ou might get a clearer understanding of the benefits
of caching if you take a
peek at the distances between
the CPU and each storage device. Here's a simplified view of
a typical motherboard with a

Pentium II microprocessor.
Notice that the level 1 (L1) cache
is invisible: It's built right into
the die (core chip) of the
Pentium II. The level 2 (L2)
cache is nearby on the microprocessor's card, but the RAM
and other devices are all much

farther away. Every additional

fraction of an inch of distance
causes cumulative delays in the
millions of transactions taking
place each second. Therefore,
the closer the caches are to
the CPU, the faster they
can supply data.

When the CPU (central processing unit) needs

a piece of data, it sends a request (a fetch) to its
nearby L1 cache, which runs at the same speed
in megahertz (MHz) as the CPU. If the L1 cache
has the data (a hit), it will send it to the CPU
almost instantaneously.

If the L1 cache doesn't have the right data (a

miss), the CPU will send a fetch to the L2 cache
controller. The cache controller checks the tag
RAM, which is a constantly changing directory
of all the lines (32-byte chunks) of data in the L2
cache's data store. The L2 cache is bigger than
the L1 cache and may run at the CPU's speed,
or half of it. If the tag RAM finds the right line of
data in the data store, the cache controller
reads the line and sends it to the CPU and the
L1 cache (in case the data is needed again very
soon).

If the data isn't found in L2 cache, the CPU sends a

fetch to the system's RAM (random-access memory). If
there's a hit in RAM (meaning the CPU found the data
there), the L2 and L1 caches will get a copy of the data as
it is sent to the CPU. The data may be stored in a particular line or group of lines in the L2 cache depending upon
which part of RAM held the data. If the L2 cache is the
write-through type, it will keep the RAM apprised of every
change the CPU makes to the data. A write-back L2
cache will perform better by only updating the RAM when
the data is getting too old to store in L2 cache any longer.

CPU with L1
cache
Pentium II

2
L2
cache

In Case Your Pipeline Bursts

If the RAM didn't have the data, the CPU is

forced to get it from the hard drive, CD-ROM, or
other storage medium. Once it finds the data, it
will drop copies in RAM, L2, and L1 cache. New
data will overwrite the memory locations of data
that has been accessed least recently.

L2 caches can send brief "bursts" of sequential data much faster

than they can send data that is scattered throughout the cache.
They can temporarily enter this burst mode because once the
first line of consecutive data is located, it doesn't take long to find
the ones right after it. Caches are divided into three groups with
different burst mode speeds

Asynchronous Burst Cache

The first L2 caches were asynchronous, meaning they operated at

their own speed, which had no relationship to the system bus (or
memory bus, from the CPU to RAM) speed. Asynchronous burst
cache was fine for systems with 25MHz or 33MHz system bus (it
wasn't called "frontside" yet) speeds, such as those found in 486
systems, but faster buses made this type of cache prone to errors

Synchronous Burst Cache

Later synchronous caches improved the speed of bursts to CPUs

by synchronizing to the system bus speed. This cut down on the
countless tiny delays caused by mismatched timing between the
cache and CPU. Classic synchronous caches are now called flowthrough caches to distinguish them from pipelined burst caches
(below).

Pipelined Burst Cache

Pipelined burst synchronous caches start sending the second line

of consecutive data before the first has been fully sent. Although
flow-through designs have a speed advantage in systems with
60MHz or 66MHz system bus speeds and in digital signal processors (DSPs) for audio applications, according to Micron
(http://www.micron.com), pipelined burst caches are better suited
to computers with 100MHz or 133MHz system bus speeds.

Compiled by Marty Sems

Graphics & Design by Jason Codr