Optical Packet-Switched

WDM Networks:
a Cost and Energy Perspective
Rodney S. Tucker
ARC Special Research Centre for
Ultra-Broadband Information Networks (CUBIN)
University of Melbourne
r.tucker@ee.unimelb.edu.au

Acknowledgements:

Jayant Baliga, Kerry Hinton, Rob Ayre, Gangxiang Shen, Wayne Sorin
Australian Research Council, Cisco

Summary

Optical packet-switched networks

Network architectures
Point-to-point WDM IP networks
IP networks with optical grooming
Optical burst switching
Optical packet switching

CAPEX and network architectures

Scaling

Energy consumption and network architectures

Core and access networks

Disclaimers:
Numbers given here are approximate - YMMV
OPEX not included

E-mail me for copies of these slides

Impact on
network growth

IP Packets

Edge
Routers

Core Router

OXC

Optical Packet-Switched WDM Network

1h

op

Is this simple model realistic?

The Reality: Map of the Internet

Patent(s) Pending and

Copyright Lumeta
Corporation 2007

Not Included:
Enterprise Networks
Content Distribution Networks
Data Centres

Number of Hops in the Internet

2006 Data

0.1

Pr [ H = k ]

0.08
0.06
0.04
0.02
0

15
10
Number of Hops, k

20

Source: P. Van Mieghem,

Performance Analysis of Computer Systems and Networks, Cambridge (2006)

25

Evolution of Optical Packet-Switched Networks

Conventional Wisdom
Capacity
CAPEX?
Energy Consumption?

Optical
Burst
Switching
(OBS)

Stage 1

Optical
Packet
Switching
(OPS)

Optical
Circuit
Switching
(OCS)
Point-to-Point
WDM (P2P)
Time

1. Point-to-Point WDM Network

Lightpath
Fiber

O/E/O Converters

Core
Router

O/E/O
Converters
Access Routers

Router provides
sub-wavelength grooming
by statistical multiplexing

2. Optical IP Network
Core Router

Edge
Routers

OXC

1h

op

2. Optical IP Network
Lightpath
Fiber

Core
Router

O/E/O Converters

OXC/ROADM

O/E/O
Converters
Access Routers

Optical bypass
Sub-wavelength grooming

3. Waveband IP Network
Lightpath
Waveband
Fiber

Core
Router

O/E/O Converters

MUX
OXC/ROADM
Waveband grooming
Optical bypass
Sub-wavelength grooming
Access Routers

Cost and Scalability of Optical Networks

Consider three network architectures:
P2P WDM

Number of users: 10 million

Parthiban et al., 2003

Optical IP

Waveband IP

Compare CAPEX

Input Parameters

Average Access rate per user: 1 - 100 Mb/s

10 million users

Capacities:
Router:
OXC:
Lightpath:
Fiber:

5 90 Tb/s
1000 ports
40 Gb/s
250 lightpaths

1000

1000
1 = 40 Gb/s

250
Component costs [Ferreira 02, Sengupta 03]
OXC, router chassis, port
Fiber, amplifiers, lightpath terminations
Ignore
Access network, search engines, data centers, etc.
Optical impairment cost e.g. regenerators
Protection & restoration
Multiple domains
Cost reductions as technology matures
OPEX

Results Network Cost

104

Cost per User ($)

P2P WDM

=
pe
o
l
S

Waveband IP
102
Optical IP

10
Todays Internet
(~ 100 kb/s)
Parthiban et al., 2003

10
Average Access Rate (Mb/s)

(Access rate per user in a busy period)

100

Sanity Check
Cumulative Capex $ Billions

90
80

Cumulative Internet Capex (North America)

150 million users

70

~$400 / user

60
50

Total

40

Core and
Metro

30

Optical transport

20
10
0

Access
2000

2005

Year

2010

Nemertes, November 19, 2007:

User demand could outpace network capacity by 2010
$137 billion global infrastructure investment needed
Based on data from Nemertes Research, 2007

Observations

Optical IP network
Can save costs in todays network
Optical bypass reduces router ports

Waveband IP network
Eliminates the bottleneck in number of ports and lightpaths
Least costly for high access rates

There is no such thing as a free lunch

If you want more bandwidth, you will have to pay for it

Energy Consumption of the Network

Hot spot

Why worry about energy consumption?

OPEX
Greenhouse Impact
Managing Hot Spots
- Getting the energy in
- Getting the heat out
Energy-limited capacity bottlenecks

Power In

Energy Model of Simple IP Network

CRS-1
~ 10 kW / rack
Packet
over
Sonet

Core

Core

Core

Fibre
Amps

12816 Edge
~ 4 kW

Metro

WDM

Edge

OLT - 100W

Passive
Optical
Network

Curb

Edge

Curb Curb

ONU ~ 5-10W

0.1 - 1000 Mb/s to the user

Baliga et al., 2007

Core

Curb

Access

Power Consumption of IP Network

25
20 hops

2008 Technology
1.0

Total
15

Todays Internet
(~ 100 kb/s)

Core

10

0.5

Access +Metro
5

SDH/WDM Links
0

Average Access Rate (Mb/s)

Baliga et al., 2007

10

% of Electricity Supply

Power (W/user)

20

Power Consumption of IP Network

10

200
2008 Technology

7.5

Power (W/user)

150

Total
5.0

100

Core
2.5

50

Access +Metro
0

20

40

60

SDH/WDM

80

Average Access Rate (Mb/s)

Baliga et al., 2007

100

% of Electricity Supply

20 hops

Observations / Questions

Access network dominates energy consumption at low rates

Standby mode?

Core network dominates at higher rates

Reduce hop count?

What is the bottleneck in the core? Speed or energy?

Optical packet switching

Optical transport (WDM) consumes relatively little energy

< 5% of energy
> 25% of CAPEX

Annual CAPEX / Annual energy OPEX > 2

Electronic Router
Forwarding Engine

Fibers
1

J
Switch
Fabric

K
F
Demutiplexers
O/E
Converters

Switch Fabrics

Buffers

Reduced bit rate

Electronics

Optics

F
Multiplexers

Power Consumption in Routers

Power consumption (W)

1,000,000
?

P = C2/3

100,000

where P is in Watts
where C is in Mb/s

10,000
10 nJ/bit
1,000

P ~ 10
100
100 nJ/bit
10
1

1 Mb/s

1 Gb/s

1 Tb/s

Router Throughput

Source: METI, 2006, Nordman, 2007

1 Pb/s

Energy per Bit in Routers

100,000

Energy per Bit (nJ)

10,000
1,000
100 nJ/bit
100
10 nJ/bit
10
?

1 nJ/bit

1
0.1

1 Mb/s

1 Gb/s

1 Tb/s

Router Throughput

1 Pb/s

Heat Load
Air Cooling Limit
1 nJ/bit

Year
Source: K. Brill, The Uptime Institute

Energy in High-End Electronic Router

Line Card
Buffer
O/E

Forwarding Engine
Switch
Fabric

Buffer

I/O

Buffer

O/E

Forwarding Engine

Switch
Control

Routing
Tables

Routing
Engine
Control Plane

Data Plane

Power
supply
inefficiency
Fans and
blowers

Energy/bit

0.7 nJ

3.2 nJ

0.5 nJ

1.0 nJ

1.1 nJ

3.5 nJ

Fraction
of Total

7%

32%

5%

10%

11%

35%

Source: G. Epps, Cisco, 2007

Network Energy Consumption per Bit

10-3
20 hops
Energy per bit (J)

10-4
~1 J/b
Total

10-5
10-6
10-7

Core
Metro + Access
WDM Links

10-8
0.1

10

Average Access Rate (Mb/s)

100

Energy in Electronic Integrated Circuits

Cwire
CMOS Gates
CMOS IC

Energy =

Egate +

1
2

Cwire V 2

Power = Energy x Bit Rate

Diversion: Capacitance of the Core Network

Source

Destination

C2

C1

E =1
2

Energy per bit per user:

V = 2, E =

1 J
2

C V 2

C 200 nF

Capacitance
per user

50% efficiency
~ 150 million users CNetwork = 200 nF x1.5 108

C Network 30 F (North America)

C Network 300 F (Global)

Energy Consumption in Access Networks

NEC CM7710T

Access N/W
Edge Node
Cisco
12816

Cabinet
Splitter

NEC
CM7700S

Cabinet

PON
NEC VF200F6

FTTN
Zyxel
VES-1616F-34

Cisco
4503

NEC GM100

PtP
Axxcelera
ExcelMax CPE
Axxcelera
ExcelMax BTS

Baliga et al., OThT6

WiMAX

Energy Consumption in Access Networks

WiMAX
Energy per bit (J)

10-5

PtP
10-7

10-8

FTTN

10-6

PON

10
100
Average Access Rate (Mb/s)

Wireless access consumes more energy than optical access

PON FTTH is greener than FTTN
Standby mode shows significant potential (OThT6)

Baliga et al., OThT6

1000

Evolution of Optical Packet-Switched Networks

Capacity
Optical
Packet
Switching
(OPS)

Stage 2
CAPEX and
Energy Barrier

Stage 1
Point-toPoint WDM
(P2P)

Optical Burst
Switching
(OBS)

All-optical
sub-wavelength
grooming

Optical
Circuit
Switching
(OCS)

Time

Stage 2 Evolution OCS to OBS

Core Router
Edge Node

Optical Burst
Switches
Electronics
WDM Link

OXC

Edge
Router

Access
Routers
Burst Assembly Router

Optical Burst Switched Network

Optical Burst Switch
No Buffering

IP Packets (duration < s)

Data Bursts (duration < ms)

IP Packets
t
Burst
Assembly
Router

C. Qiao and M. Yoo, JHSN, 1999

Headers
Offset

Optical Burst Switching

Source

Header

Destination

Offset

Time

Burst
length
< ms

D
A
T
A

Switch

4
Distance

Normalized Cost: OBS vs. IP

20

Optical IP

Normalized Cost

1 Gbit/s Access Rate

15

Without wavelength conversion

OBS

10
With wavelength conversion
5

Waveband IP

10-6

10-5

10-4

10-3

10-2

Average Path Blocking Probability

Parthiban et al., OFC 2005

10-1

Why is OBS more Costly?

OBS
MUX

Wavelengths

Waveband
Bursts

Wavelengths

Bursts

DEMX
Waveband

Requires increased number of lightpaths for a given blocking probability

Switch technology requires fast reconfiguration time: More costly per

port than slow OXCs (MEMS etc.)

Solution: Waveband Burst Switching (WBS)

Waveband Route

Burst

WBS
Waveband

Waveband

Pros:
Requires fewer OXC ports
Cons:
OXC ports must be wideband
Requires waveband (i.e. multi-channel) wavelength conversion
Dispersion issues
Y. Huang et al., OFC 2004, Pathiban et al., OFC 2006

Waveband Burst Switching (WBS)

1 Gbit/s Access Rate
7
Normalized Cost

Optical IP
5

WBS with wavelength conversion

Waveband IP

WBS (WC) + Deflection Routing +

Burst Segmentation

10-5

Parthiban et al., OFC 2006

10-4

10-3

10-2-2
10-1-1
10
10
Average Path Blocking Probability

Evolution of Optical Packet-Switched Networks

Capacity

Stage 3

Stage 2

Stage 1
Point-toPoint WDM
(P2P)

Optical
Burst
Switching
(OBS)

Optical
Packet
Switching
(OPS)

Optical
Circuit
Switching
(OCS)

Time

Optical Packet-Switched Network

The Holy Grail of Optical Networking

Optical Packet
Switches
With Buffering

OXC

Access Router

Will a viable optical buffering technology emerge?

Can OPS reduce energy consumption?

Optical Packet Switch (OPS)

Forwarding
Engine

Electronics

Optics

F
Demutiplexers

WavelengthInterchanging
Cross Connect +
Header
Replacement

Output Buffers

Input Synchronizers

Mutiplexers

Optical Buffer Structures

Control

Delay Line

Delay

Variable
Delay Line

Delay

Recirculating
Loop

Cross Point

Staggered
Delay Line

Delay

Cross Point

Delay

Cross Point

Optical Fiber Buffers

Total buffer capacity of 104 Gigabits
~ 103 RAM chips
Cost < US$ 50k
Buffer power dissipation < 1 kW
Cisco CRS-1 with 1000 ports,
250 ms buffering per port

Total fibre length = 40 Gm

150 times distance from Earth to Moon!
.

OPS with 1000 ports,

250 ms buffering per port
optical fiber delay lines

Reduced Buffer Size

Minimum feasible
buffer size?

~5 s (~20 packets) buffering per port

Optical fiber delay lines

Total fibre length = 1000 km

Loss per port = 0.2 dB

Planar waveguide or
slow light delay lines
(0.1 dB/cm)

Loss per port = 33,000 dB

Loss
Tucker, PS 2007

Energy

Holographic Buffers
Holographic Medium:
Photorefractive material
(e.g LiNbO3, polymers)
Output image
Write speeds up to 1 Gb/s
Read speeds up to 10 Gb/s
Reference beam
Input image

Orlov et al., Proc IEEE, 2004

Psaltis, CLEO 2002
Ashley et al. IBM J. Res. Dev., May 2000

Storage
density

Retention
time

Access
Time

Write Time

1/3

~ 50 s

> 500 s

Comparison of Optical Buffer Technologies

Technology

Fiber

Planar WG,
Slow Light

Optical
Resonator

Holographic

CMOS

Access
Time

Structuredependent

Structuredependent

Small

~ 50 s

200 ps

Retention
Time

> 500 s

< 5 s

1-100 ns

64 ms

Capacity
(Packets)

> 2,000

< 20

<< 1

Energy/bit

~ 1 fJ

~ 1 pJ

~ 1 pJ

~ 1 pJ

~ 1 fJ

Physical
Size

Very
Large

Medium

Medium

Small

Very
Small

Chirp
Sensitivity

No

Small

Large

Large

No

Show stopper
Challenge
Tucker, PS 2007

Packet Switch Cross-Connect Technologies

Projected Energy Consumption
AWG-Based

CMOS

~5 pJ/bit

~10 pJ/bit

3-stage Clos
AWGs

Wavelength
(1.5 pJ/bit)
Converters

SOA-Based

Optical
Interconnects
(1 pJ/bit)
>100 pJ/bit
(10 pJ/bit per gate)

Benes Array

Tucker, JLT 2005, OSN 2008

Benes

Chips
(2 pJ/bit)

Observations on Optical Packet Switching

No viable optical buffering technology in sight

Optical switch fabrics may become competitive with CMOS

Not clear whether optical packet switching will solve the energy
bottleneck problem

Summary

Optical bypass reduces CAPEX and energy consumption

- Promising future for optical cross-connects and ROADMs

Energy bottleneck in routers is looming

- More significant than the so-called electronic speed bottleneck

Can optical packet switching overcome the energy bottleneck?

- Optical buffering is currently a show-stopper

Think Energy per Bit