Advanced Computer

Architecture
(CS5413)

Department of Computer Science

Capital University of Science and Technology (CUST)
Week 2

2
Outline
• Trends in Power and Energy in Integrated Circuits
• Dependability
• Measuring, Reporting and Summarizing Performance
• Quantitative Principles of Computer Design

3
Outline
• Trends in Power and Energy in Integrated Circuits
• Dependability
• Measuring, Reporting and Summarizing Performance
• Quantitative Principles of Computer Design

4
Energy is the biggest challenge
today
• Today, energy is the biggest challenge facing the
computer designer for nearly every class of
computer.
• First, power must be brought in and distributed
around the chip
• Second, power is dissipated as heat and must be
removed.
• From the viewpoint of a system designer, there are
three primary concerns

5
1- Maximum Power
• First, what is the maximum power a processor ever requires?
• Meeting this demand can be important to ensuring correct
operation.
• For example, if a processor attempts to draw more power
than a power-supply system can provide (by drawing more
current than the system can supply), the result is typically a
voltage drop, which can cause devices to malfunction.
• Modern processors can vary widely in power consumption
with high peak currents; hence they provide voltage indexing
methods that allow the processor to slow down and regulate
voltage within a wider margin.
• Decreases voltage decreases performance. (discussed later)
6
2- Sustained Power Consumption
• Second, what is the sustained power consumption?
• This metric is widely called the thermal design power
(TDP) because it determines the cooling requirement.
• TDP is neither peak power, which is often 1.5times
higher, nor is it the actual average power that will be
consumed during a given computation, which is likely
to be lower still.
• A typical power supply for a system is typically sized
to exceed the TDP, and a cooling system is usually
designed to match or exceed TDP.

7
Managing Heat
• Modern processors provide two features to assist in
managing heat, since the highest power (and hence
heat and temperature rise) can exceed the long-
term average specified by the TDP.
• First, as the thermal temperature approaches the
temperature limit, circuitry lowers the clock rate,
thereby reducing power.
• Should this technique not be successful, a second
thermal overload trap is activated to power down
the chip.

8
Throttle Stop
• ThrottleStop is a piece of software that monitors and adjusts
three different types of CPU throttling and tests processor
performance
• https://www.ultrabookreview.com/31385-the-throttlestop-
guide/

9
BD PROCHOT
• Short for Bi-directional Processor Hot.
• PROCHOT is an emergency throttling method triggered
when a CPU hits its maximum temperature.
• Can be used by manufacturers to lower the performance
when needed.
• Activate it when the battery is below a certain threshold.
• Previously we could disable it using software like Throttle
Stop. Seems like it is not possible for modern systems.

10
3- Energy Efficiency
• The third factor that designers and users need to
consider is energy and energy efficiency.
• Power is simply energy per unit time
• In general, energy is always a better metric because
it is tied to a specific task and the time required for
that task.
• In particular, the energy to complete a workload is
equal to the average power times the execution
time for the workload.

11
Energy and Power Within a
Microprocessor
• For CMOS (complementary metal oxide semiconductor)
chips, the traditional primary energy consumption has
been in switching transistors, also called dynamic
energy.
• The dynamic energy required per transistor is
proportional to the product of the capacitive load
driven by the transistor and the square of the voltage:

• This equation is the energy of pulse of the logic

transition of 0→1 → 0 or 1 → 0 → 1. The energy of a
single transition (0 → 1 or 1 → 0) is
12
Power Required by a Transistor
• The power required per transistor is just the product of the
energy of a transition multiplied by the frequency of transitions

• Dynamic power and energy are greatly reduced by lowering the

voltage, so voltages have dropped from 5 V to just under 1 V in
20 years.
• For a fixed task, slowing clock rate reduces power, but not
energy.
• The capacitive load is a function of the number of transistors
connected to an output and the technology, which determines
the capacitance of the wires and the transistors.
13
Example

14
Relation between Voltage and
Switching Frequency [Wikipedia]
• MOSFET (MOSFET stands for metal-oxide-semiconductor field-effect
transistor) based digital circuits operate using voltages to represent logical
state. The voltage at these nodes switches between a high voltage and a
low voltage during normal operation—when the inputs to a logic
gate transition, the transistors making up that gate may toggle the gate's
output.
• Toggling a MOSFET's state requires changing its gate voltage from below
the transistor's threshold voltage to above it (or from above it to below it).
• Higher supply voltages result in faster slew rate (rate of change of voltage
per unit of time) when charging and discharging, which allows for quicker
transitioning through the MOSFET's threshold voltage.
• Additionally, the more the gate voltage exceeds the threshold voltage, the
lower the resistance of the transistor's conducting channel is which
allows for quicker charging and discharging of the capacitance of the
subsequent logic stage.
• Quicker transitioning afforded by higher supply voltages allows for
operating at higher frequencies. 15
More Transistors and faster
switching means increased power
• The increase in the number of transistors switching and
the frequency with which they change dominate the
decrease in load capacitance and voltage, leading to an
overall growth in power consumption and energy.
• The first microprocessors consumed less than a watt, and
the first 32-bit microprocessors (such as the Intel 80386)
used about 2 W, whereas a 4.0 GHz Intel Core i7-6700K
consumes 95 W.
• Given that this heat must be dissipated from a chip that is
about 1.5 cm on a side, we are near the limit of what can
be cooled by air, and this is where we have been stuck for
nearly a decade.
16
What is the effect of all this?
• Given the preceding equation, you would expect
clock frequency growth to slow down if we can’t
reduce voltage or increase power per chip.
• This has indeed been the case since 2003
• Note that the period of flatter clock rates
corresponds to the period of slow performance
improvement range

17
18
How to improve Energy
Efficiency
• Modern microprocessors offer many techniques to
try to improve energy efficiency despite flat clock
rates and constant supply voltages.

19
1-Do Nothing Well
• Most microprocessors today turn off the clock of
inactive modules to save energy and dynamic
power.
• For example, if no floating-point instructions are
executing, the clock of the floating-point unit is
disabled.
• If some cores are idle, their clocks are stopped.

20
2- Dynamic voltage-frequency
scaling (DVFS)
Devices have periods of low activity where there is
no need to operate at the highest clock frequency
and voltages (i.e. highest clock rate frequency).
Modern Processors Typically operates offer few clock
frequencies and voltage in which to operate that use
lower power and energy.
.

21
Design for Typical Cases
Given that PMDs and Laptops are often idle, memory
and storage offer low power modes to save energy.
DRAMs have a series of increasingly low power
modes to extend battery life in PMD and Laptop.

22
Overvolting and Undervolting
(Wikipedia)
• In computer architecture, dynamic voltage scaling is
a power management technique in which the voltage used
in a component is increased or decreased, depending upon
circumstances.
• Dynamic voltage scaling to increase voltage is known
as overvolting; dynamic voltage scaling to decrease voltage
is known as undervolting.
• Undervolting is done in order to conserve power
• Overvolting is done in order to support higher frequencies
for performance.

23
3-Design for the typical case.
• Given that PMDs and laptops are often idle, memory and
storage offer low power modes to save energy.
• For example, DRAMs have a series of increasingly lower
power modes to extend battery life in PMDs
• Microprocessors for PCs have been designed instead for
heavy use at high operating temperatures, relying on on-
chip temperature sensors to detect when activity should be
reduced automatically to avoid overheating.
• This “emergency slowdown” allows manufacturers to design
for a more typical case and then rely on this safety
mechanism if someone really does run programs that
consume much more power than is typical.
24
4-Overclocking
• Intel started offering Turbo mode in 2008, where
the chip decides that it is safe to run at a higher
clock rate for a short time, possibly on just a few
cores, until temperature starts to rise.
• For example, the 3.3 GHz Core i7 can run in short
bursts for 3.6 GHz.
• For single-threaded code, these microprocessors
can turn off all cores but one and run it faster.

25
Feature Size
• Integrated circuit processes are characterized by
the feature size, which is the minimum size of a
transistor or a wire in either the x or y dimension.
• Feature sizes decreased from 10 μm in 1971 to
0.016 μm in 2017; in fact, we have switched units,
so production in 2017 is referred to as “16 nm,” and
7 nm chips are underway.
• The density of transistors increases with a decrease
in feature size.

26
More Transistors mean more
power
• Increasing the number of transistors increases power
even if they are idle
• Also current leakage increases in processors with
smaller transistor sizes.
• As a result, very low-power systems are even turning
off the power supply to inactive modules in order to
control loss because of leakage.
• In 2011 the goal for leakage was 25% of the total
power consumption, with leakage in high-
performance designs sometimes far exceeding that
goal.
27
The Shift in Computer Architecture
Because of Limits of Energy
• The importance of power and energy has increased
the scrutiny on the efficiency of an innovation, so
the primary evaluation now is tasks per joule or
performance per watt.
• As transistor improvement decelerates, computer
architects must look elsewhere for improved
energy efficiency.
• This has led architects to reexamine the
fundamentals of processors’ design in the search
for a greater energy-cost performance.

28
Comparison of the energy of arithmetic
operations and energy cost of accesses to SRAM
and DRAM

29
Some observations
• 32-bit floating-point addition uses 30 times as
much energy as an 8-bit integer add.
• A 32-bit DRAM access takes 20,000 times as much
energy as an 8-bit addition.
• A small SRAM is 125 times more energy-efficient
than DRAM, which demonstrates the importance of
careful uses of caches and memory buffers.

30
Domain-specific processors

• The new design principle of minimizing energy per task

combined with the relative energy and area costs have
inspired a new direction for computer architecture
• Domain-specific processors save energy by reducing wide
floating-point operations and deploying special-purpose
memories to reduce accesses to DRAM.
• Although such processors perform only a limited set of tasks,
they perform them remarkably faster and more energy
efficiently than a general-purpose processor.
• Computers in this energy-aware world will likely be
combinations of general-purpose cores that can perform any
task and special-purpose cores that do a few things extremely
well
31
Outline
• Trends in Power and Energy in Integrated Circuits
• Dependability
• Measuring, Reporting and Summarizing Performance
• Quantitative Principles of Computer Design

32
Dependability
• Some fault are widespread, where many can be
limited to single component in module.
• Utter failure of module at one level may be
considered only a component error in higher-level
module.
• This distinction is helpful in trying to find ways to
build dependable computers.

33
Service Accomplishment and
Interruption
• How can we define when a system is operating properly.
• This theoretical point became concrete with the popularity of
Internet services.
• Infrastructure providers started offering service level agreements
(SLAs) or service level objectives (SLOs) to guarantee that their
networking or power service would be dependable.
• For example, they would pay the customer a penalty if they did not
meet an agreement of some hours per month.
• SLA could be used to decide whether the system was up or down.
• 1. Service accomplishment, where the service is delivered as specified.
• 2. Service interruption, where the delivered service is different from the
SLA.
• Transitions between these two states are caused by failures (from
state 1 to state 2) or restorations (2 to 1).
34
Module Reliability

• It is the measure of continuous service accomplishment (or, time to

failure) from a reference initial point.
• MTTF (mean time to failure) is reliable measure.
• For example, if Brand X’s car engines average 500,000 hours before they
fail completely and have to be replaced, 500,000 would be the engines’
MTTF.
• The calculation is used to understand how long a system will typically last.
• You calculate it by adding up the total operating time of the products
you’re assessing and dividing that total by the number of devices.
• For example: Let’s say you’re figuring out the MTTF of light bulbs. How
long do Brand Y’s light bulbs last on average before they burn out? Let’s
further say you have a sample of four light bulbs to test. Light bulb A lasts
20 hours. Light bulb B lasts 18. Bulb C lasts 21. And bulb D lasts 21 hours.
That’s a total of 80 bulb hours. Divided by four, the MTTF is 20 hours.
35
36
Problems
• What happens when we’re measuring things that don’t fail quite
as quickly?
• For those cases, though MTTF is often used, it’s not as good of
a metric. Because instead of running a product until it fails,
most of the time we’re running a product for a defined length of
time and measuring how many fail.
• For example: Let’s say we’re trying to get MTTF stats on Brand
Z’s tablets. They test 100 tablets for six months. Let’s say one
tablet fails exactly at the six-month mark. So, we multiply the
total operating time (six months multiplied by 100 tablets) and
come up with 600 months. Only one tablet failed, so we’d divide
that by one and our MTTR would be 600 months, which is 50
years. And so the metric breaks down in cases like these.
37
Example from Toshiba
• “The MTTF is a statistical value that defines after
how much time a first failure in a population of
devices may occur (measured in hours). If MTTF is
given as 1 million hours, and the drives are operated
within the specifications, one drive failure per hour
can be expected for a population of 1 million
drives. ”

• https://www.toshiba-storage.com/trends-technolo
gy/mttf-what-hard-drive-reliability-really-means/
38
MTTR
Mean Time to Repair (MTTR) is a key performance metric used to
measure the average time it takes to complete maintenance or repairs
on equipment, machinery, or other systems 1
It is an important indicator for evaluating the efficiency of different
processes that can influence operations 1
• MTTR encompasses the following steps Alerting the technicians
about the issue.
• Diagnosing the root cause of the problem.
• Undertaking necessary repairs.
• Giving the equipment time to cool down, if necessary.
• Reassembling, aligning, and calibrating the equipment.
• Finally, setting up, testing, and restarting the equipment for regular
production.

39
MTBF
• MTBF (mean time between failures) is the average time between
repairable failures of a technology product.
• The higher the time between failure, the more reliable the system.
• To calculate, take the data from the period you want to calculate
and divide that period’s total operational time by the number of
failures.
• So, let’s say we’re assessing a 24-hour period and there were two
hours of downtime in two separate incidents. Our total uptime is
22 hours. Divided by two, that’s 11 hours. So our MTBF is 11
hours.
• Because the metric is used to track reliability, MTBF does not
factor in expected down time during scheduled maintenance.
Instead, it focuses on unexpected outages and issues.
40
Module Availability
• It is measure of the service accomplishment with
respect to the alternation between the two stages
of the accomplishment and interruption.
• For non redundant system with repair, module
availability is
=

41
Example

42
Outline
• Trends in Power and Energy in Integrated Circuits
• Dependability
• Measuring, Reporting and Summarizing Performance
• Quantitative Principles of Computer Design

43
Measuring, Reporting and
Summarizing Performance
• When we say one computer is faster than another one is, what do we
mean?
• The user of a cell phone may say a computer is faster when a program runs
in less time
• An Amazon.com administrator may say a computer is faster when it
completes more transactions per hour.
• The cell phone user wants to reduce response time—the time between the
start and the completion of an event— and the operator of a WSC wants to
increase throughput—the total amount of work done in a given time.
• In comparing design alternatives, we often want to relate the performance
of two different computers, say, X and Y.
• The phrase “X is faster than Y” is used here to mean that the response time
or execution time is lower on X than on Y for the given task. In particular, “X
is n times as fast as Y”

44
Measuring, Reporting and
Summarizing Performance Contd.

45
Elapsed Time and CPU Time
• The most straightforward definition of time is called wall-clock time,
response time, or elapsed time, which is the latency to complete a task,
including storage accesses, memory accesses, input/output activities,
operating system overhead—everything.
• With multiprogramming, the processor works on another program while
waiting for I/O and may not necessarily minimize the elapsed time of
one program.
• CPU time recognizes this distinction and means the time the processor
is computing, not including the time waiting for I/O or running other
programs.

46
Benchmarks
• One of the most successful attempts to create
standardized benchmark application suites has
been the SPEC (Standard Performance Evaluation
Corporation), which had its roots in efforts in the
late 1980s to deliver better benchmarks for
workstations.
• There are now SPEC benchmarks to cover many
application classes.
• All the SPEC benchmark suites and their reported
results are found at http://www.spec.org

47
Some links
• https://www.spec.org/benchmarks.html

• https://www.spec.org/cpu2017/Docs/install-guide-
windows.html

48
Comparing Two Computers
• We could normalize execution times to a reference computer by
dividing the time on the reference computer by the time on the
computer being rated, yielding a ratio proportional to performance.
• SPEC uses this approach, calling the ratio the SPECRatio.
• For example, suppose that the SPECRatio of computer A on a
benchmark is 1.25 times as fast as computer B; then we know

• Notice that the execution times on the reference computer drop out
and the choice of the reference computer is irrelevant when the
comparisons are made as a ratio

49
50
51
More links
• https://www.intel.com/content/www/us/en/gamin
g/resources/read-cpu-benchmarks.html

• https://www.cpubenchmark.net/

52
Outline
• Trends in Power and Energy in Integrated Circuits
• Dependability
• Measuring, Reporting and Summarizing Performance
• Quantitative Principles of Computer Design

53
Amdahl’s Law
• “the overall performance improvement gained by optimizing a single
part of a system is limited by the fraction of time that the improved
part is actually used”
• Amdahl's law is often used in parallel computing to predict the
theoretical speedup when using multiple processors.
• For example, if a program needs 20 hours to complete using a single
thread, but a one-hour portion of the program cannot be parallelized,
therefore only the remaining 19 hours’ execution time (p=0.95) can
be parallelized
• Then regardless of how many threads are devoted to a parallelized
execution of this program, the minimum execution time cannot be
less than one hour.
• Hence, the theoretical speedup is limited to at most 20 times the
single thread performance
54
The Processor Performance
Equation
• CPU time for a program can then be expressed as

• We can also count the number of instructions executed

(instruction count or IC)
• If we know the number of clock cycles and the instruction
count, we can calculate the average number of clock cycles
per instruction (CPI).

• That means
• CPU Clock cycles for a program = CPI x Instruction count
• Hence
55
The Processor Performance
Equation Contd.
• Hence processor performance is dependent upon three
characteristics:
• instruction count.
• clock cycles per instruction
• clock cycle time (or rate)
• Furthermore, CPU time is equally dependent on these three
characteristics; for example, a 10% improvement in any one
of them leads to a 10% improvement in CPU time.
• Expanding the first formula into the units of measurement
shows how the pieces fit together

56
Improving Performance
• Unfortunately, it is difficult to change one
parameter in complete isolation from others
because the basic technologies involved in
changing each characteristic are interdependent:
• Luckily, many potential performance improvement
techniques primarily enhance one component of
processor performance with small or predictable
impacts on the other two.

57
That is all for Week 2

58