You are on page 1of 30

Guided Image Filtering

Kaiming He
Jian Sun
Xiaoou Tang

The Chinese University of Hong Kong


Microsoft Research Asia
The Chinese University of Hong Kong

Introduction
Edge-preserving filtering
An important topic in computer vision
Denoising, image smoothing/sharpening, texture decomposition, HDR
compression, image abstraction, optical flow estimation, image superresolution, feature smoothing

Existing methods
Weighted Least Square [Lagendijk et al. 1988]
Anisotropic diffusion [Perona and Malik 1990]
Bilateral filter [Aurich and Weule 95], [Tomasi and Manduchi 98]
Digital TV (Total Variation) filter [Chan et al. 2001]

Introduction
Bilateral filter

qi

W ( p) p

jN ( i )

ij

spatial Gs(xi-xj)

input p

bilateral
W=GsGr
range Gr(pi-pj)

output q

Introduction
Joint bilateral filter [Petschnigg et al. 2004]

qi

W ( I ) p

jN ( i )

ij

bilateral filter: I=p

spatial Gs(xi-xj)

input p

bilateral
W=GsGr

output q

range Gr(Ii-Ij)

E.g. p: noisy / chrominance channel


I: flash / luminance channel
guide I

Introduction
Advantages of bilateral filtering
Preserve edges in the smoothing process
Simple and intuitive
Non-iterative

Introduction
Problems in bilateral filtering
Complexity
Brute-force: O(r2)
Distributive histogram: O(logr) [Weiss 06]
Bilateral grid: band-dependent [Paris and Durand 06], [Chen et al. 07]
Integral histogram: O(1) [Porikli 08], [Yang et al. 09]

Approximate
(quantized)

Introduction
Problems in bilateral filtering
Complexity

Example: detail enhancement

Gradient distortion

gradient
reversal

Preserves edges,
but not gradients

gradient
reversal

input

enhanced

Introduction
Our target - to design a new filter
Edge-preserving filtering
Non-iterative
O(1) time, fast and non-approximate
No gradient distortion

Advantages of bilateral filter

Overcome bilateral filters


problems

Guided filter
qi pi ni

min (aI i b pi ) 2 a 2
( a ,b )

ni - noise / texture
Linear regression
input p

qi aI i

output q

qi aI i b

cov( I , p )
a
var( I )
b p aI

guide I
Bilateral/joint bilateral filter does
not have this linear model

Definition

Guided filter
Extend to the entire image
In all local windows k ,compute
the linear coefficients

cov k ( I , p )
ak
vark ( I )
bk pk aI k

Compute the average of akIi+bk in


all k that covers pixel qi

(a I

k |i

k i

ai I i bi

qi
2

qi

bk )

Definition

Guided filter

cov k ( I , p )
ak
vark ( I )

Parameters
Window radius r

bk pk aI k

Regularization

qi

(a I

k |i

k i

ai I i bi

qi
2

2r
3

bk )

Guided filter: smoothing


a cascade of
mean filters

cov( I , p )
var( I )

var(I )
cov( I , p )

b p aI

input p

qi a I i b p

output q

var(I )
guide I

a0
b p

r : determines
band-width
(like s in BF)

Guided filter: edge-preserving


qi a I i b

qi a I i I i a b
: degree of

cov( I , p )
a

var(I )

edge-preserving
(like r in BF)

qi

I i

guide I

output q

Example edge-preserving smoothing

input &
guide

guided
filter
(let I=p)
r=4, =0.12

r=4, =0.22

r=4, =0.42

s=4, r=0.1

s=4, r=0.2

s=4, r=0.4

bilateral
filter

Our target - to design a new filter


Edge-preserving filtering
Non-iterative
O(1) time, fast and non-approximate
No gradient distortion

Advantages of bilateral filter

Overcome bilateral filters


problems

Definition

Complexity
mean, var, cov in all local windows

cov k ( I , p )
ak
vark ( I )

Integral images [Franklin 1984]

bk pk aI k

O(1) time independent of r

qi ai I i bi

Non-approximate

O(1) bilateral
(32-bin, 40ms/M)
[Porikli 08]

O(1) bilateral
(64-bin, 80ms/M)

O(1) guided
(exact, 80ms/M)

Gradient Preserving
bilateral filter

input

guided filter

filtered

q a I
large
fluctuation
detail
(input - filtered)

enhanced
(detail * 5 + input)

gradient
reversal

Example detail enhancement

gradient
reversal

bilateral filter

input (I=p)

bilateral filter
s=16, r=0.1

guided filter

guided filter
r=16, =0.12

Example detail enhancement

gradient
reversal

bilateral filter

input (I=p)

bilateral filter
s=16, r=0.1

guided filter

guided filter
r=16, =0.12

Example HDR compression

gradient
reversal

bilateral filter

keep
anti-aliased

guided filter

input HDR

bilateral filter
s=15, r=0.12

guided filter
r=15, =0.122

Example flash/no-flash denoising


gradient
reversal

input p
(no-flash)

joint bilateral filter


s=8, r=0.02

joint bilateral

guide I
(flash)

guided filter
r=8, =0.022

guided filter

Beyond smoothing
Applications: feathering/matting, haze removal

very small
guide I

preserve most
gradients

output q

q a I
input p

Example feathering

guide I
(size 3000x2000)

Example feathering

filter input p (binary segmentation)

Example feathering

filter output q (alpha matte)

Example feathering

guide I

filter input p

filter output q
0.3s
image size 6M

matting Laplacian
[Levin et al. 06]
2 min

Example haze removal

guide I

filter input p
(dark channel prior
[He et al. 09])

filter output q

Example haze removal

guide I

guided filter
(<0.1s, 600x400p)

global optimization
(10s)

Limitation
What is an edge inherently ambiguous, context-dependent
weaker
edge

halo

halo

stronger
texture

Input

Bilateral filter
s=16, r=0.4

Guided filter
r=16, =0.42

Conclusion
We go from BF to GF
Edge-preserving filtering
Non-iterative
O(1) time, fast, accurate
Gradient preserving
More generic than smoothing

Thank you!