Tensor Processing Unit

PRESENATATION
 DEDICATION
It’s Start in the name of Allah Almighty, who is over creator and my
strong pillar, my source of inspiration, wisdom, knowledge and
understanding. He Has-been the source of my strength and on His
wings only have I soared.
I dedicate this Report to the founder of Shakta Khoum.
“Imran Khan Niazi”
“May ALLAH PAK grant him a long life”
 ACKNOWLEDGEMENT
First of all I would like to express
my special thank to our special Prof.
UMAIR SHAFIQ Who gave us the
opportunity to work on this
presentation. By doing this
presentation we got enriched with many
information which can help us in
future again special thanks to our
respected .
 Group Members
NAMES ROLL NUMBER

MUHAMMAD ADNAN G1F18BSCS0029

CHANGAIZ PRINCE G1F18BSCS0027

 Introduction
A tensor processing unit (TPU) is an
AI accelerator application-specific
integrated circuit (ASIC) developed
by Google specifically for neural
network machine learning,
particularly using Google's own
TensorFlow software. Google began
using TPUs internally in 2015, and
in 2018 made them available for
third party use, both as part of its
cloud infrastructure and by offering
a smaller version of the chip for
sale.
 Products
Header text TPUv1 TPUv2 TPUv3 Edge v1
Date
Introduced 2016 2017 2018
Process Node 28nm 20nm? 12nm?
Die Size
(mm2) 331 ? ?
On chip 28 ? ?
memory (MiB)
Clock Speed 700 ? ?
(MHz)
Memory (GB) 8GB DDR3 16GB HBM 32GB HBM
TDP (W) 40 200 250
TOPS 23 45 90
 First generation TPU
The first-generation TPU is an 8-bit matrix
multiplication engine, driven with CISC
instructions by the host processor across
a PCIe 3.0 bus. It is manufactured on a 28
nm process with a die size ≤ 331 mm2.
The clock speed is 700 MHz and it has
a thermal design power of 28–40 W. It has
28 MiB of on chip memory, and 4 MiB of 32-
bit accumulators taking the results of a
256×256 systolic array of 8-bit multipliers.
[8] Within the TPU package is 8 GiB of dual-
channel 2133 MHz DDR3 SDRAM offering 34 GB/s
of bandwidth. Instructions transfer data to or
from the host, perform matrix multiplications
or convolutions, and apply activation
functions.
 Second generation TPU
The second-generation TPU was announced in
May 2017.[10] Google stated the first-
generation TPU design was limited by memory
bandwidth and using 16 GB of High Bandwidth
Memory in the second-generation design
increased bandwidth to 600 GB/s and
performance to 45 teraflops. The TPUs are
then arranged into four-chip modules with a
performance of 180 teraflops. Then 64 of
these modules are assembled into 256-chip
pods with 11.5 petaflops of performance.
Notably, while the first-generation TPUs
were limited to integers, the second-
generation TPUs can also calculate
in floating point.
 Third generation TPU

The third-generation TPU was

announced on May 8, 2018.Google
announced that processors themselves
are twice as powerful as the second-
generation TPUs, and would be
deployed in pods with four times as
many chips as the preceding
generation. This results in an 8-
fold increase in performance per pod
(with up to 1,024 chips per pod)
compared to the second-generation
TPU deployment.
 Edge TPU
In July 2018, Google announced the
Edge TPU. The Edge TPU is Google's
purpose-built ASIC chip designed to
run machine learning (ML) models
for edge computing, meaning it is
much smaller and consumes far less
power compared to the TPUs hosted
in Google datacenters (also known
as Cloud TPUs). In January 2019,
Google made the Edge TPU available
to developers with a line of
products under the Coral brand. The
Edge TPU is capable of 4 trillion
operations per second while using
2W.
 Pixel Neural Core
On October 15, 2019,
Google announced
the Pixel 4 featuring
the Pixel Neural
Core, which contains
an instantiation of
the Edge TPU
architecture.
 History
Google announced its Tensor Processing Units (TPU for short), in May 2016
as a Custom made Application Specific Integrated Circuits for Machine
Learning. It’s built to be compatible with Google’s TensorFlow project,
which is an open source software library for machine learning across a
range of tasks. It can build and train neural networks to detect and deal
cipher correlation, in a manner that’s lucid to human patterns and
reasoning. Some of it’s widely publicized uses include:
• RANK BRAIN — Machine learning AI system that powers Google search engine.
• GOOGLE MAPS — Where used to improve the accuracy of results. Example:
Reading Street numbers
• ALPHA GO — Program developed by Deep mind to play Go.
To really understand how exciting TPUs are, however, we need a little bit
of background, and which is why I’m going to give you a brief overview of
both CPUs and GPUs, both of which are types of electronic circuitry used
in computer systems.
 Advantages of TPU
• Using TPUs offers a number of advantages
in terms of increasing efficiency and
speed of computation, including the
following benefits:
• Accelerated performance of linear algebra
computation, which is used heavily in
machine learning applications.
• Minimized time-to-accuracy when training
large, complex neural network models:
models that previously took weeks to
train can converge in hours on TPUs.
• Scalable operations across different
machines with their TPU servers.
 Disadvantage of TPU
• TPUs can’t be found in local markets easily. They may be
expensive and probably limited to work for only one
operating systems directly (but accessible through the
internet from any node interconnect).
How different is a TPU ,CPU and GPU?
 Performance
As a comparison, consider this:

• CPU can handle tens of operation per cycle

• GPU can handle tens of thousands of operation per

cycle

• TPU can handle up to 128000 operations per cycle

 Purpose
• Central Processing Unit (CPU): A processor designed to solve every
computational problem in a general fashion. The cache and memory design is
designed to be optimal for any general programming problem.

• Graphics Processing Unit (GPU): A processor designed to accelerate the

rendering of graphics.

• Tensor Processing Unit (TPU): A co-processor designed to accelerate deep

learning tasks develop using TensorFlow (a programming framework); Compilers
have not been developed for TPU which could be used for general purpose
programming; hence, it requires significant effort to do general programming
on TPU
 Manufacturers
• Central Processing Unit
(CPU): Intel, AMD,
Qualcomm, NVIDIA, IBM,
Samsung, Hewlett-Packard,
VIA, Atmel and many others

• Graphics Processing Unit

(GPU): NVIDIA, AMD,
Broadcom Limited,
Imagination Technologies
(PowerVR)

• Tensor Processing Unit

(TPU): Google
 Usage
• Central Processing Unit (CPU): General purpose programming
problem

• Graphics Processing Unit (GPU): Graphics rendering, Machine

Learning model training and inference, efficient for programming
problem with parallelization scope, General purpose programming
problem

• Tensor Processing Unit (TPU): Machine Learning model (only in

TensorFlow model) training and inference
If-you-have-any-question-please-keep-it-for-
yourself.