1

Notes:

P|V oc| P|V oc|

P|V oc | P|V oc |
For a specific document l (Vl ), i
, i,j can be re-written as i l , i,j l (V ocl
is the words used in document l), because we only need to care about words
that occurred in document l .

Empirical First and Second Moments

= E[x1 ] =

|Docs|
X 1
1
cl
|Docs|
|Vl |
l=1

= E[x1 x2 ]
P airs

0 + 1

|Docs|
X
1
0
1
(cl cl diag(cl ))

|Docs|
|Vl |(|Vl | 1)
0 + 1
l=1

Empirical Third Moment and Its Linear and

Quadratic Forms

T riples
= E[x1 x2 x3 ]

0
02
(E[x1 x2 ]+E[x1 x2 ]+E[x1 x2 ])+
()
0 + 2
(0 + 1)(0 + 2)

|Docs|
X
1
1
[cl cl cl
|Docs|
(|Vl |)(|Vl | 1)(|Vl | 2)
l=1

cli (ei ei ei )

cli clj (ei ei ej )

cli clj (ei ej ei )

i,j

i,j

|V oc|

|V oc|

|V oc|

|V oc|

+2

cli clj (ej ei ei )]

i,j

|Docs|
|V oc|
X
X
1
1
0
[cl cl +cl cl +cl cl
cli (ei ei +ei ei +ei ei )]
|Docs|
(|Vl |)(|Vl | 1) 0 + 2
i
l=1

02
( )
(0 + 1)(0 + 2)

T riples(1,
1, ) =

|Docs|
X
1
1
|Docs|
(|Vl |)(|Vl | 1)(|Vl | 2)
l=1

[cl cl ( cl ) + 2diag(cl )

cTl diag(cl )

mul

(cl ) cl cl (cl )]

|Docs|
X
1
1
0

[(cl cl diag(cl ))( T )+(cl cl diag(cl ))T + T (cl cl diag(cl ))]

|Docs|
(|Vl |)(|Vl | 1) 0 + 2
l=1

02
( T )( )
(0 + 1)(0 + 2)

T riples(1,
, ) =

|Docs|
X
1
1
|Docs|
(|Vl |)(|Vl | 1)(|Vl | 2)
l=1

[cl ( T cl )2 + 2(cl cl ) 2(cl )(cTl ) cl (cTl ( ))]

|Docs|
X
1
1
0
[( T )[(cTl )cl (cl )]+[cl (cTl )(cl )](T )+( T cl )2 T (cl )]
|Docs|
(|Vl |)(|Vl | 1) 0 + 2
l=1

02
( T )2
(0 + 1)(0 + 2)