CANH L NH T NG V C LNG DI CHUYN

Phm Th Bo Khoa Ton Tin hc Trng HKKHTN Tp.HCM

Ni dung
Trn c s im nh
Tm kim bng Brute Force Tm kim bng Gradient (Motion Estimation) Lucas-Kanade Lucas Kanade

Trn c s c trng
im trng yu (interest points) SIFT Brown & Lowe, Ton cnh

Phm Th Bo

Canh l nh

Lm sao canh l t ng 2 nh? Hai hng tip cn:

C s c trng
Tm vi c trng trong c 2 nh Tnh ton l

C s im nh
Tm l m hu ht im nh gn nhau
Phm Th Bo

Canh l
Tm kin bng Brute Force l tip cn n gin nht (c thm)
Cn xc nh hm so khp nh
SSD (Sum-of-Squared Difference) , tng quan chun, so cnh, ...

Tm vi tt c tham s trong phm v ca mnh:

V d cho tnh tin:

for tx=x0:step:x1, for ty=y0:step:y1, so snh image1(x,y) to image2(x+tx,y+ty) end; end end;

Cn chn chnh xc x0,x1 v step

Chuyn g xy ra nu step qu ln?
Phm Th Bo

Canh l (brute force)

Chng ta mun g?
wx' a wy' = d w g b e h c x f y i 1

for a=a0:astep:a1, for b=b0:bstep:b1, for c=c0:cstep:c1, for d=d0:dstep:d1, for e=e0:estep:e1, for f=f0:fstep:f1, for g=g0:gstep:g1, for h=h0:hstep:h1, so snh image1 to H(image2) end; end; end; end; end; end; end; end;
Phm Th Bo

Vn ca Brute Force
Khng thc
phc tp O(N8) Gi tr bt u/kt thc v bc nhy khng r rng

Chng ta c th lm g?
Dng thp Cho mi trng hp c bit (ton cnh b xoay), c th gim cn O(N4):
H = K1R1R2-1K2-1 (4 DOF: f v quay)

Cch khc: gradient trn hm li

Lm sao dng c lng li gim? C th dng thp Tng t nh cng c c lng di chuyn (motion estimation)

Phm Th Bo

c lng di chuyn: Optical flow

Bt u c lng mi im nh ring, xem nh di chuyn ca c nh

Phm Th Bo

Ti sao c lng ?
L do:
Theo vt (track) hnh vi i tng Chnh xc camera L nh (khm nh) Ti cu trc vt th 3D Tc ng c bit

Phm Th Bo

nh ngha bi ton: optical flow

Lm sao c lng s di chuyn im nh t nh H vo I?

Gi quyt bi ton tng ng im nh
Ly mt im nh trong H, H xem cc im nh gn mu trong I

Gi thit
n nh mu: mt im trong H ging trong I
Vi nh xm, y l xm

Di chuyn t: cc im khng di chuyn qu xa

y gi l bi ton optical flow

Phm Th Bo

Optical flow (nh xm)

Xem xt
xm: Q: ci no bng nhau? Di chuyn t: (u v v nh hn 1 pixel)
Gi s dng khai trin chui Taylor ca I:

Phm Th Bo

Phng trnh Optical flow

Kt hp hai phng trnh

Trong gg gii hn u v v tin v zero, , nn tr thnh

Phm Th Bo

Phng trnh Optical flow

Q: c bao th khng bit v c bao nhiu phng trnh cho h mi pixel? i l? Trc quan, bit g?
Xc nh c hng ca gradient Khng bit thnh phn song song cnh

Phm Th Bo

Bi ton khe h

Phm Th Bo

Bi ton khe h

Phm Th Bo

Gii bi ton khe h

bao nhiu phng trnh cho mt pixel?
tng: thm rng buc
Thng thng nht, gi s rng trn ti cc b Mt phng thc: xem nh cc pixel lng ging ging (u,v)
Nu dng d ca s 5x5 5 5 th cn 25 ph hng trnh t h cho h mt pixel! i l!

Phm Th Bo

Vi RGB
Nu nh mu RGB th phi cn 25*3 phng trnh cho mt pixel!

Phm Th Bo

Lukas-Kanade flow
Bi ton: c nhim phng trnh hn bin

Gii quyt: gii ma trn vung

Ti thiu:

Tng tt c pixel trong ca s K x K K thut ny c Lukas & Kanade trnh by (1981)

Phm Th Bo

iu kin gii
Ti u (u, v) tha mn phng trnh Lucas-Kanade

Khi no gii c?
ATA kh nghch ATA khng qu nh, t nhiu Gi tr t ring i 1 v 2 ca ATA khng kh c qu nh h T A A tt 1/ 2 khng qu ln(1 = gi tr ring ln hn)

Cnh

Cc gradient ln, cho tt c 1 ln, 2 nh

Phm Th Bo

Vng kt cu thp

gradient c ln nh
1, 2 nh
Phm Th Bo

10

Vng kt cu cao

gradient khc bit, ln s ln

1, 2 ln
Phm Th Bo

Li trong Lukas-Kanade
Kh nng no dn n li?
Do gi thit ATA kh nghch Do gi thit t nhiu trong nh

Khi no gi i thit gp vn
sng bin ng Di chuyn xa Mt im khng ging lng ging
Ca s ln qu Kch thc bao nhiu?

Phm Th Bo

11

Xem xt gi thit di chuyn t

C nh?
Chc chn l khngiln hn 1 pixel Lm sao gii quyt bi ton?
Phm Th Bo

Gim phn gii!

Phm Th Bo

12

c lng t th n mn optical flow

u=1.25 pixels u=2.5 pixels

u=5 p pixels

image H Thp Gauss nh H

u=10 pixels

image I I image Thp Gauss nh I

Phm Th Bo

c lng t th n mn optical flow

Thc hin lp L-K Bin i Thc hin lp L-K

. . .

image nh H J
Thp Gauss nh H
Phm Th Bo

nh I I image Thp Gauss nh I

13

Tnh tin
Hn na, mnh c th ch c tnh tin trong (u,v) M hnh di chuyn khc no?
quay, affine, lut xa gn

Tng t c th xem xt Jacobi :

A T A = JI( I)T J T A b = J T I t ( I)T

T i
Phm Th Bo

Canh l nh

Mc tiu: c lng tnh tin n (u,v)

Trn c s thp vi Lukas-Kanade
Phm Th Bo

14

Canh l da trn c trng

1. Tm vi vi c trng quan trng (im trng yu) 2. So chng qua 2 nh 3. Xem xt bin i

Chn c trng
Chn nhng im gc cnh (nh ra) Lm sao chn?
Vi ca s c gi tr ring ln

Gi l Harris Corner Detector

Phm Th Bo

Xc nh c trng

Phm Th Bo

15

So khp c trng
Mt kh nng:
So khp cc c trng tm thy trong nh 1 vi c trng tm thy trong nh 2
V d SSD ca cc mnh nh xung gq quanh mi c trng g

Dng cc so khp thnh cng c lng homography

To li quan h

Cc bi ton:
Lm g khi mt vi mnh ca vi im trng yu ging nhau?
Tng kch thc mnh

Lm g khi vi mnh cng cng c trng m nhn khc nhau do t l, quay,

dng Lucas-Kanade vi m hnh di chuyn affine Cch tt hn: Bin i c trng bt bin t l (Scale-Invariant Feature Transform - SIFT)
Phm Th Bo

c trng bt bin
Schmid & Mohr 1997, Lowe 1999, Baumberg 2000, Tuytelaars & Van Gool 2000, Mikolajczyk & Schmid 2001, Brown & Lowe 2002, Matas et. al. 2002, Schaffalitzky & Zisserman 2002

Phm Th Bo

16

c trng bt bin cc b
Dng cc c trng cc b m bt bin khi tnh tin, quay, t l, v cc tham s nhkhc

B m t c trng

Phm Th Bo

Cc ng dng
im c trng dng:
Canh l nh (homography, ma trn c bn) Ti to 3D Theo vt di chuyn Nhn dng i tng Truy tm d liu v ch mc Chun ng robot

Phm Th Bo

17

Harris corner detector

C.Harris, M.Stephens. A Combined Corner and Edge Detector. 1988

Phm Th Bo

tng

Dng ca s nh xc nh cc im Di chuyn ca s ra cc hng m thy c s thay i ln v cng

Phm Th Bo

18

Harris Detector: tng

Vng ph V hng: khng thay i trong tt c cc hng

cnh: h khng thay i dc theo hng cnh

Phm Th Bo

gc: thay i dng k trong cc hng

Harris Detector: ton hc

Thay i cng vi di chuyn [u,v]:

E (u , v) = w( x, y ) [ I ( x + u, y + v) I ( x, y )]
x, y

Hm ca s Hm ca s w(x,y) =

Cng thay i

Cng

hay 1 trong ca s, 0 ngoi

Phm Th Bo

Gauss

19

Harris Detector: ton hc

Di chuyn nh[u,v] chng ta xp x theo:

u E (u, v) [u , v ] M v
M l ma trn 22 matrix tnh t o hm nh:

I x2 M = w( x, y ) x, y Ix I y

IxI y 2 Iy

Harris Detector: ton hc

Phn loi im ca nh bng gi tr ring ca M:

cnh 2 >> 1

gc 1 v 2 ln, 1 ~ 2; E tng trong cc hng

1 v 2 nh; E gn nh hng
cc hng

Vng phng
Phm Th Bo

cnh 1 >> 2 1

20

Harris Detector: ton hc

o mc gc:

R=

det M Trace M

det M = 12 trace M = 1 + 2

Phm Th Bo

Harris Detector
Thut ton:
Tm cc im vi gc ln R (R > ngng) Ly cc im cc i a phng ca R

Phm Th Bo

21

Harris Detector: v d

Phm Th Bo

Harris Detector: v d
Tnh gc vi R

Phm Th Bo

22

Tm cc im m: R>threshold

Ly cc im cc i da phng ca R

23

Phm Th Bo

Harris Detector: tnh cht

Bt bin quay

Ellipse quay nhng hnh dng (gi tr ring) tng t Gc R bt bin khi quay nh
Phm Th Bo

24

Harris Detector: tnh cht

Bt bin trong mt phn bin i affine

Ch dng o hm => bt bin khi cng thay i I I + b T l cng : I a I R

Ngng

x (ta nh)
Phm Th Bo

x (ta nh)

Harris Detector: tnh cht

Nhng: khng bt bin khi xy ra t l!

Cc im nh cnh
Phm Th Bo

gc !

25

Xc nh bt bin t l
Xem xt cc vng (vng trn) c kch thc khc nhau quanh mt im Cc vng c kch thc tng ng s nhn ging nhau trong cc nh

Phm Th Bo

Xc nh bt bin t l
Bi ton: lm sao chn c cc vng trn tng ng c lp trong g mi nh? Chn t l gc tt nht

Phm Th Bo

26

Chn c trng
Phn b cc im trong nh l nh nhau

Phm Th Bo

p dng khng c chn ti

M t: c nh s c trng ca mi nh
Want evenly distributed spatially Sp xp cc im bng bn knh khng c chn ti a [Brown, Szeliski, Winder, CVPR05]

Phm Th Bo

27

B m t c trng
bit cch xc nh cc im. Cu hi k tip: Lm so sao khp chng?

?
M t im: 1. Bt bin 2. c bit
Phm Th Bo

B m t bt bin cho quay

Tm hng cc b

Hng tri ca gradient

Trch cc mnh quan h hng ny

Phm Th Bo

28

Cc mnh c nh hng a t l
im trng yu
Cc gc Harris a t l Hng t gradient khng r Bt binhnh hc cho quay

Vector m t
Ly mu chun ha dc ca mnh cc b (8x8) Bt bin sng vi thay i affine trong cng

[Brown, Szeliski, Winder, CVPR2005]

Phm Th Bo

Vector m t
Hng = gradient khng r Frame bt bin quay
V tr (x, y, s) + hng ()

Phm Th Bo

29

B m t cho a t l

Phm Th Bo

Vector m t MOPS
Mnh c hng 8x8
Mu mc t l 5

Chun ha dc: I = (I )/

8 pixels

Phm Th Bo

30

Khp c trng

Khp c trng
Tm mi vi mi ci
Mi c trng trong mt nh, xem xt tt cc cc c trng khc trong nh kia

Bm
Tnh mt m t ngn cho mi vector c trng, hoc bm b m t

K thut lng ging gn nht

Cy-kd v bin th

Phm Th Bo

31

Khng gian c trng

Khng so khp tt c cc c trng, chn s lng tng t khp? Lm sao lm?
SSD(patch1,patch2) SSD( t h1 t h2) < ng ng Lm sao c ngng?

Phm Th Bo

KHng gian c trng

Cch tt hn [Lowe, 1999]:
1-NN: SSD ca khp gn nht 2-NN: SSD khp hai mnh gn nht 1 NN tt hn 2-NN 1-NN 2 NN C phi ci t nht ta c s tt hn nhng ci cn li?

Phm Th Bo

32

Khng gian c trng

C th tnh H t cc im xanh?
khng! V vn cn qu nhiu ngoi Lm sao?

Phm Th Bo

Khp c trng

Lm sao bit so khp khng tt?

Phm Th Bo

33

Pht sinh ngu nhinmu tt nht (RAndom SAmple Consensus RASAC)

Select count Chn one mt match, khp, m sinliers ni bt

Phm Th Bo

Bnh phng ti thiu

Tm vector tnh tin trung bnh

Phm Th Bo

34

RANSAC cho c lng homography

RANSAC lp: 1. Chn cc cp c trng 4 (ngu nhin) 2. Tnh homography H (chnh xc) 3. Tnh s tri vi SSD(pi, H pi) < 4. Gi tp ni tri ln nht 5. Tnh li bnh phng ti thiu c lng H vi tt c tri

Phm Th Bo

RANSAC

35

c trng SIFT
c trng bt bin
Thit lp frame bt bin
Tm t l ti u
Cc i/cc tiu khng gg gian t l DOG x, , y, s

Tm hng ti u
Cc i phn b ca cc gradient cc b

T m t vector
Histogram cc gradient cc tt tt 128 chiu

c trng SIFT l
Bt bin hnh hc to bin i t ng t,
Mnh cho bin i affine

Bt bin sng cho bin i affine trong cng

Phm Th Bo

Ti sao nhn dng trong Panoramas?

Phm Th Bo

36

RANSAC vi Homography

RANSAC vi Homography

37

RANSAC vi Homography

M hnh xc sut

38

Tm panoramas

Phm Th Bo

Tm panoramas

Phm Th Bo

39

Tm panoramas

Phm Th Bo

Tm panoramas

40

Homography cho quay

Tham s ha mi camera bng quay v tiu c

cho cp homographies

Phm Th Bo

Chnh theo b

Phm Th Bo

41

Phm Th Bo

Trn a tn
Burt & Adelson 1983
Trn a tn s

Phm Th Bo

42

Kt qu

Phm Th Bo

43