Professional Documents
Culture Documents
Li Video Mining 2003
Li Video Mining 2003
Uploaded by
Gokulesh DanapalCopyright
Available Formats
Share this document
Did you find this document useful?
Is this content inappropriate?
Report this DocumentCopyright:
Available Formats
Li Video Mining 2003
Li Video Mining 2003
Uploaded by
Gokulesh DanapalCopyright:
Available Formats
!
" #
$
%& $ ' ( ( " "
)
# ' * ' +
, "
, -.-
!
"
#
$ "!
%
&
% %
'
(
$ )!
*
#
!"#$
%
& '
( )
"#$*+ ,--. )
"#$* ,---+ /
011,
&
2
& 2
Æ
3
2
4 5
2
6
)7 ,--*
0111 2
011,+
/
2
8
8
/
9
3
2
& 2
:
2
&
2
2
2
2
;
2 /
<
/
8
2
)
2
+
/
<
2
/
2
0 =
>
? @
2
A
2
/
B
2
2
#
* /
2
.
C
C
8 /
2
/
/
D
2
2
/
<
4 7 )4 ,--. 7 ,--B+
8
D
8 )D ,--A+
2
D
5
2
/ 5 )4 ,-B.+
2
C
E
2 <
)7 ,--*+
2
5
%3 C 3 C 3 C( 3
)
0111+
D
5
/
2
<
2
2 / 2
/ ;
6
3 2
/
2
2
7 D DFC@ &DC 34' )&
,--- G ,---+ 4
2
<
) 2
011,+
2
2
2
)= 011,+
:
5 )= 0110+ 2
/
/ 2
<
2 /
/
2
2
/
<
Æ
C 2
;
/
2
5
/
Æ
3 2
/
2
6
2
2
>
2
;
/
2
;
C
/
2
2
/ 2
!
2 5
: 2
6
2
:3
)"6
,--B+
/
8 &
F
)
,--*+
2
:
5
) 2 0111+
) 2
0110+
)
0110+
2
D
2
2
2
<
2 /
2
5
2
2 8
2
E ,+ 2
;
>
2
6
2
2
9 0+
2
2
2
2
2
/
2
2 /
/
9
<
/
2
! "# $
/
2 /
2
6
)= H 011?+ 3
-0AI
--I
2
2
<
/
2
/
/
E
)J 011,+
<
/
5
3
..I /
2
<
,1
&
D" = )D" = ,--.+
.AI
D
C
6
2
/
% &
2
) ,-**+ /
Æ
/ 2
/
5
E
3 6
6
)
2 2 ,-.B+ %
(
/ 2
8
2
E 02
2
<
,,
)4 ,-B.+ "
<
,, )+
02
)+
2
)
2
+ # /
2
)+
3
C
E ,+ 2
29 0+ 2
2
9 ?+
/ 9
@+
#
ABC
AB
AB AC BC
A B A C C
A B B
)+ )+
/"* +
, ! "
,
&! ,!
,
& -!
% ! '
(
/
3
/
"
2
2
$
/
&
2
D
2
2
' ; 2
) +
?
;
K ,@ ) ) + L ) +
L ) + L ) ++
) +
#
;
Æ
K,
K,
),,+
K, K,
Æ <
/
) % (+
,
,
M
/
; 2 C
D
2 2
% ! ' )#
2
/ E
<
; 2 L 0 L @ L B
/
; 2 2
5
2
, <
2
<
; 2
L0 L@ L* L,1 5 0 0 ? ?
0 &
2;
5
2
0A
1A
2
Movie1 Movie2
11
10
10
9
9
8
8
7 7
Variance
Variance
6 6
5
5
4
4
3
3 2
2 1
0
2 4 6 8 10 12 14 16 2 4 6 8 10
)+ )+
Mean Mean
/"* ! -
, ! #
,! #
"
? $
2
H
H
<
H
5
8
D
% (
<
,0
3
2
%
! ($
3
2
6
6
6
2
<
; 2 L0 L@ L*
L,; 2 L, L? L.
L .
3
,
2
02
5
2
2
2
0
2
2
2
2
2
2
5
2
? 3
%%
*
(
'
E
,+ E
2
8 )
+
M
02
9 0+ E
5
2
#
2
2
/ 5/
2
'
<
?
/
9
/
02
/
2
2
02
+
<
,?
2
8
2E ),+
/ )0+
2
)?+
)@+ 2
)A+ )3:+ 2
/ )B+
2
3 /
< /
M
;
!
2
2
C
6
/
2
3: / #
2
< / 2
;
6
/
C /
?
/
+ *
, -
&
.
2
<
,? <
,@)+
Initial Speaker
Environ. Snd. Initial speaker
Audio Shot-based Modeling models
Stream Silence
Audio AV-based
Music
Shot Classification Speech Speech Speaker Speakers
Sequence Speech Segmentation Clusters Identification
&
Clustering
Updated speaker models Identified
Speakers
Face Detection, Recognized Unsupervised
Visual Shot Recognition
& Talking faces Speaker Model
Stream Detection Shot sequence Mouth Tracking Adaptation
/"* &
3 5
5
5
)
+
)+ )+ )+
/"* 0 , ! ,!
(
,!
/
2
)
2
+
;
;
'
/
5
K N O
N1 ,O
/
/
2
/
5
2
M
5
5
3
2
) + ) +
5
) +
D
)= 011?+
K ) +
½ ¾
),0+
K L ) +
½ ¾
) +
<
,@)+
2
6
2
5
<
,@)+ 3
<
,@)+ <
) +
2
5
2
) +
) +
<
,A
2
5
<
2 2 "
6
5
2 9
2 5 2 5
+
<
8
2
/"* #
2
)
+
2
;
/
3 0
2
E ,+
/
5
M
/
)= 011?+9 0+ @
)= J 011,+
<
,B "
3
2 !
!
2
C
&
)C &+
)& ,--.+
C &
8
K 6
5
P ) + K 0, ) Q Q
E >= T & (Count ++) < L
Leaving
E >= T & Count > L Silence E >= T
(Output the beginning (Count = 0)
point of a speech
segment)
E<T&
Count < L
In
In
E >= T E<T
Silence
Speech
E >= T &
Count < L
E < T & Count > L
E<T (Output the endpoint
(Count = 0) Leaving
of a speech segment)
Speech
E < T & (Count ++) < L
/"* 1
'
Q + 0, ) L ,0 ) L ,++
!
Q Q Q
;
5 ,
!
3
C &
P ) +
2
M
) + E ) + K
P ) +
K < ) +
1
9
"
+
/
2
<
,?
/
/
/ 2
#
#
A0, ;
$
)$+
#
M
$
+% / #0
3 2
/ /
9 ,@
5
Æ )
4 ,--A+
5 < 2 ) 9 +
2
2
$
! C $
2
;
)4 ,--A+
$ 2
2 "
$
K Q K
,
%
Q
& & %
M $
2
K Q
5
K ,
%
2
) 9 Q + )
,-*-+
'
) 9 Q + K 0 0 Q 0 )Q + 0 ) R +Q ) R +
'
(
)
*
),?+
R
;
C /
2
K N O
*
N1 ,O
$ $ $
5
2 ) 9 Q +
$
/
2
'
++ (
/ 2
/ 2
) +
A, A0 A@ /
2
*
5
+ *
5 ;
;
+ +
D
2
+
M
2
;
) + K
) N O L N O+
%,-)
),@+
$
; 2
2
$
, 1 &
5
+1 ' &
M
2
; 2
.
2
/
2
;
) 9 + K ,
2
2 / 2
%2 (
; 2
2
E
!
" #
;
" #$ & C &
;
.
'
.
" $
;
/ ) Q + ) Q + ;
$
;
)
2 011,+
K
L L L ),A+
Q K L Q L L Q L ) L + ) +) + ),B+
5
Q D
;
.
) ?0
2+
.
$
" %$ F
;
.
" $
3"
2
/
2
$
/
;
;
.
K
L RL L ),*+
/
/ /
/ % (
,1 1
2
/
/
K ) + ),.+
%
) +
< R
/
%
R K ) ) + + ),-+
%
%
F2
3"
3"
.
"% &
3"
6
6
M
3"
) +
#5 ,.
1
%
, M
& K
) +
%
),,1+
< #5 ,A ,B
;
3
.
2
, $
?0
5
1 #0 &
<
@ 2
2
2
E ,+ &
5
;
9 0+
/ 2
;
/
8
#
1 *
<
01
#
2
K ),,,+
2
5 ,
9
5 1 C
3 01
E
01
1?
1 ? 1 B
2
1B
01 2
2 4
"
*2
K ),,0+
*
*2
5 ,
9
1
/
* * 3 01
01
% "
2
C
*
2
2
*
N1,O
*
6
*
2
*
/
'
#
)= H 011?+ <
N1,O
#
/
(
/
2
.
&
/
) *
E 02
2
C
/
3
, , ,
K
, ,
#
,
, , ,
)
5 B
+
<
K 1 @ K 1 , ? K 1
,
/
1@ ?
, , ,
1 , ?
02
1 & 2
3
2
;
3
KN O <
;
% (
. % % %
% ( % ( %
( <
%1 ?(
%
. %
( 3
1 ? 1 % 02
( ?
%
%
( <
2
$
.
0
0 K
#
.
K
, , , %
K
, ,
,
, , , %
K
L L
, % ),,?+
, %
%
(
%3M'( <
K 1 0 K
K ,
, % 3 , %
K 1
/
, %
<
, %
, % , % , % ,
/
K ,
1 /
% , %
< %
(
, % , %
K ,
%
M
B B
, %
/ 2
2
2
1,
22
/ )
,--1+
8 ),,@+
5 ,
2
1
2
22
/
2
2
2
<
/ 2
M
2 / 2
/
6
! 2
C
1
F
2
"% '
(
6
& 5
2
"
E ,+
2
;
2
9 0+
8
2 6
6
<
"%
&
2
2 /
2 /
2
,
,
2 2
5 5
2
5
&
, 0A
5 2
5
2
2
5
2
=3 ) /
33+ 5
)= 011,+
/ =3
/ ;
"%% # ) C
/
2
3
)C 2 011,+ ;
3
E )&
+ )
+ + )
+
>
>
>
2 <
3 -
3
-
<
"#$,
0--*
!
6
/
, )%* ,
-(
+ 0 )%.
/ "(
+ ? )%0 (
+ #
,
'
8 /
;
6
/
#
,,
?
.1 3
"
"
K L 4 K L
3
.?I
4
02
, /
2
2
5 2
?
2
2
3
-
2
/
D
0
2 *
#
#
" #
)
#
. + /
- 0 1 -
2 '1 -
* 3 #
1
4 / 3 #
1
4 /
#
5 6 6 667 667 5 6 867 667
"
9 6 667 897 9 ) 9:7 897
#
" . - ;
- 0 1 -
2 '1 -
* 3 #
1
4 / 3 #
1
4 /
#
: 6 6 667 667 < " 6 667 :"7
"
5 6 6 667 667 5 6 " 887 667
#
) .
- 0 1 -
2 '1 -
* 3 #
1
4 / 3 #
1
4 /
#
< 6 667 8)7 < 6 667 8)7
"
) 6 6 667 667 ) 6 ) 8 7 667
0 -A0
3
-1I
/
2
4
2 .?I
..I
@0A 5 D
2
,,AI
2 Æ
/
,0
2
3 C &
3; C; &; %F2 (
2
)3; C+
3; 2
C
/
)<3+
1
)<4+ 2
) 3+
"
+2 K
6
+# K
6
# K, + 2
,0)+ /
3
*A?I 3 00?I <3
4
3"
,0)+
*.BI 3 0,I <3
)+
,0)+
:
3
.0I 3 01I <3
3"
2
/
C
8
E )+
)+
'
!
Æ
C
2
6 /
2
/
?0 B@
<
,*)+ 3
?0 $
<
/
<
,*)+ &
/
<
,.
2
/
/
2
2
2
2
/
2
3 C
2
2
2 2
/
2
; C
2 !
#
"
, !
,! #4
,! =
& - > 1/ ?
@ ""8 5" 9 )" "97 :57
&@ ): "8 )< "5 "<7 :<7
-@ 6 A < 9 ")7 ::7
> 6 < 5 88
1 "67 A7 "87
, !
& - > 1/ ?
@ ")A "< "" "" ""7 :87
&@ <A )6" < "67 867
-@ 6 8 : < ""7 :87
> A 9 : :<
1 ":7 57 ""7
,!
& - > 1/ ?
@ "59 "A ) "6 "67 867
&@ 5 ): ) 9 97 857
-@ 6 6 ") : 87 8"7
> 8 "" 5 9)
1 ""7 57 "57
,!
Identification Accuracy Comparison Identification Accuracy Comparison
0.787 0.82
0.82 0.753
0.663 0.65
0.793
0.787
0.47
0.7665
0.753 0.756
1 2 3 1 2 3
The applied approaches The applied approaches
)+ )+
32-component 64-component w/face cue w/o face cue
/"* 3 ?
#4
=
, ! )"
95
,!
)
+
2
/
3
C
3
2
/
2 2 < /
/
AA 3 /
6
3 & -
Æ 5 2
5
/
2
A
),A+
,
%
( A
%
(E
,+
E % 5 2
8
2
(9 0+
E % 5
(9 ?+
E %
>
(9 @+
E %
2
(9 A+
E % 2
52
(9 B+
E % 2
2 (
, 0
,?
2 >
B
#
=
58 6"
59" 6 :
55 6)9
2
5< 6""
B
5: 6 <
=
5 < 6:8
<
C
/ 2
;
5
2 2
>
S p e e c h - S i l e n c e S e p a r a t i o n U s i n g a d a p t i ve s i l e n c e d e t e c t o r
0 .2
0 .1 5
0 .1
0 .0 5
-0 .0 5
-0 .1
Speakers – Ground Truth
-0 .1 5
-0 .2
0 0 .2 0 .4 0 .6 0 .8 1 1 .2 1 .4 1 .6 1 .8 2
S a m p l e In d e x 5
x 10
Likelihood-based speaker identification result
Talking face detection result
Final speaker identification result
Legends:
: A’s non-talking face : B’s non-talking face
: A’s talking face : B’s talking face
: Speaker A : Speaker B
/"* .
2
52
5 D
2 2
2
/
<
2
4 !
3 2
:
) 2
+
3
/
8
!
5
! 8
2 <
"#$*
&
M < #
4
&
&
3
M
##&-A0-,A0
D "2
&
2
2 D" = " 3 &3
-(
C 2 C * ! "
$ " "
- *! 3
< "
011,
& 4
3# "
%'
2
3 '( &3 ";-- " ,---
& $ 2
" % 2
#
&
& ' &
: C
&
( "
'34"3 C
M
F
2 ,--.
D
3$
3 % : <
: E
"
8( "
333
< &
=
: ,--A
D" = & : $
% D" < '
4 =
( F
; $ 4
:
00 '
,--.
D G = J 7 %
3 :
<
& : ( & ";-.
,--.
G # % 2 S 3
&
<
'
2
( #
;-- ,---
= 7 M
H && % / 2
' F 3 &( &3 ";10
0110
= ' $ H '
M %"
/
: "
( G
#
,1)@+E-?1-?. 011,
= 7 H && 4
!
& 5
6
"' F
&
011?
= 7 M
H && %3 2
/
3 : & <
& 3( "
"
4 =
4
: &
011?
= T J G J T = & %3 4 4
#
'
#
M
<
3 4 3
#
( &3 ";1, 011,
= < H
G H && %3 ' &
<
: 3 "2
/
( &3 ";1, 011,
H H G C G 3
"
' ,-*-
" 7
$
4
&
M7 ,--1
2 & % 3 D 4 & E 3
F/ <
2( ###
3 "
-)@+E?@0?A* 011,
G /
*
5 -$ * *
, /
*
- M 7
2
F
"
,-.0
"#$ 45
$
%"#$* & 8
4
( ' !"#$ M0.B, "#$ :
G ,---
"6
=
4 <
#6
%3
' 3
( G
: &
4
*)@+E?@A?A? '
,--B
4 ' 4 4 %4 2
/
F $
2
( ###
3 "
?),+E*0.? ,--A
4 H
$ * *
- 8 M 7
2E
D D "
,-B.
4 7 D
%&
<
: ( 3& G
*)A+E?A-?B. ,--.
H %: 2
&
&
= F
5( "
### &
: "
4
**A*., ,--*
D & < %'
&
<
F 3
(
3&
;11
' 4 M
0111
2 2 3 " * 9 5+
4 3
F
"
,-.B
2
& 3
3 " ' ' #G %3
:
F
( "
" # @B*BE?*,
?.0 G
0110
2 & = " ' %:
E 3 D
3
$
:
( &#;11
M 7
2 0111
2
" %& : "
C 3
( ###
&
: ,,)@+EA00A?A 011,
C= = &7
G4 %:
"
<
"
'( "
" # @B*BE?A-
?*1 G
0110
7 7 C= = C %#
F <
=
"
<
: C
M ( ### "
0-B?1A ,--B
7 7 C= %: & &
&
<
' =
3 ( "
" # ?100E@AA.
<
,--*
J H && %3 & 3 <
3
'
( ###
3
"
-)@+E@@,@A* 011,
You might also like
- The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeFrom EverandThe Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeRating: 4 out of 5 stars4/5 (5807)
- The 7 Habits of Highly Effective People Personal WorkbookFrom EverandThe 7 Habits of Highly Effective People Personal WorkbookRating: 4 out of 5 stars4/5 (2515)
- The Iliad: The Fitzgerald TranslationFrom EverandThe Iliad: The Fitzgerald TranslationRating: 4 out of 5 stars4/5 (5646)
- The Odyssey: (The Stephen Mitchell Translation)From EverandThe Odyssey: (The Stephen Mitchell Translation)Rating: 4 out of 5 stars4/5 (7771)
- The Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeFrom EverandThe Subtle Art of Not Giving a F*ck: A Counterintuitive Approach to Living a Good LifeRating: 4.5 out of 5 stars4.5/5 (20047)
- How to Win Friends and Influence People: Updated For the Next Generation of LeadersFrom EverandHow to Win Friends and Influence People: Updated For the Next Generation of LeadersRating: 4 out of 5 stars4/5 (2315)
- Pride and Prejudice: Bestsellers and famous BooksFrom EverandPride and Prejudice: Bestsellers and famous BooksRating: 4.5 out of 5 stars4.5/5 (20479)
- Never Split the Difference: Negotiating As If Your Life Depended On ItFrom EverandNever Split the Difference: Negotiating As If Your Life Depended On ItRating: 4.5 out of 5 stars4.5/5 (3285)
- Wuthering Heights Complete Text with ExtrasFrom EverandWuthering Heights Complete Text with ExtrasRating: 4 out of 5 stars4/5 (9955)
- The Picture of Dorian Gray (The Original 1890 Uncensored Edition + The Expanded and Revised 1891 Edition)From EverandThe Picture of Dorian Gray (The Original 1890 Uncensored Edition + The Expanded and Revised 1891 Edition)Rating: 4 out of 5 stars4/5 (9054)
- Art of War: The Definitive Interpretation of Sun Tzu's Classic Book of StrategyFrom EverandArt of War: The Definitive Interpretation of Sun Tzu's Classic Book of StrategyRating: 4 out of 5 stars4/5 (3321)
- Anna Karenina: Bestsellers and famous BooksFrom EverandAnna Karenina: Bestsellers and famous BooksRating: 4 out of 5 stars4/5 (7503)
- American Gods: The Tenth Anniversary EditionFrom EverandAmerican Gods: The Tenth Anniversary EditionRating: 4 out of 5 stars4/5 (12949)
- Habit 3 Put First Things First: The Habit of Integrity and ExecutionFrom EverandHabit 3 Put First Things First: The Habit of Integrity and ExecutionRating: 4 out of 5 stars4/5 (2507)
- Habit 1 Be Proactive: The Habit of ChoiceFrom EverandHabit 1 Be Proactive: The Habit of ChoiceRating: 4 out of 5 stars4/5 (2556)
- The 7 Habits of Highly Effective PeopleFrom EverandThe 7 Habits of Highly Effective PeopleRating: 4 out of 5 stars4/5 (2568)
- Wuthering Heights (Seasons Edition -- Winter)From EverandWuthering Heights (Seasons Edition -- Winter)Rating: 4 out of 5 stars4/5 (9486)
- The Iliad: A New Translation by Caroline AlexanderFrom EverandThe Iliad: A New Translation by Caroline AlexanderRating: 4 out of 5 stars4/5 (5719)
- How To Win Friends And Influence PeopleFrom EverandHow To Win Friends And Influence PeopleRating: 4.5 out of 5 stars4.5/5 (6529)
- Habit 6 Synergize: The Habit of Creative CooperationFrom EverandHabit 6 Synergize: The Habit of Creative CooperationRating: 4 out of 5 stars4/5 (2499)
- The Picture of Dorian Gray: Classic Tales EditionFrom EverandThe Picture of Dorian Gray: Classic Tales EditionRating: 4 out of 5 stars4/5 (9758)
