Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Save to My Library
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Additive Update Algorithm for Nonnegative Matrix Factorization

Additive Update Algorithm for Nonnegative Matrix Factorization

Ratings: (0)|Views: 19 |Likes:
Published by ijcsis
Nonnegative matrix factorization (NMF) is an emerging technique with a wide spectrum of potential applications in data analysis. Mathematically, NMF can be formulated as a minimization problem with nonnegative constraints. This problem is currently attracting much attention from researchers for theoretical reasons and for potential applications. Currently, the most popular approach to solve NMF is the multiplicative update algorithm proposed by D.D. Lee and H.S. Seung. In this paper, we propose an additive update algorithm that has faster computational speed than the algorithm of D.D. Lee and H.S. Seung.
Nonnegative matrix factorization (NMF) is an emerging technique with a wide spectrum of potential applications in data analysis. Mathematically, NMF can be formulated as a minimization problem with nonnegative constraints. This problem is currently attracting much attention from researchers for theoretical reasons and for potential applications. Currently, the most popular approach to solve NMF is the multiplicative update algorithm proposed by D.D. Lee and H.S. Seung. In this paper, we propose an additive update algorithm that has faster computational speed than the algorithm of D.D. Lee and H.S. Seung.

More info:

Published by: ijcsis on Oct 10, 2012
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

10/10/2012

pdf

text

original

 
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 10, No. 9, September 2012
Additive Update Algorithm for Nonnegative MatrixFactorization
Tran Dang Hien
Vietnam National Universityhientd_68@yahoo.com Do Van TuanHanoi College of Commerce and TourismHanoi – Vietnamdvtuanest@gmail.com Pham Van AtHanoi University of Communicationsand Transportphamvanat83@vnn.vn
 Abstract
—Nonnegative matrix factorization (NMF) is anemerging technique with a wide spectrum of potentialapplications in data analysis. Mathematically, NMF can beformulated as a minimization problem with nonnegativeconstraints. This problem is currently attracting much attentionfrom researchers for theoretical reasons and for potentialapplications. Currently, the most popular approach to solve NMFis the multiplicative update algorithm proposed by D.D. Lee andH.S. Seung. In this paper, we propose an additive updatealgorithm, that has faster computational speed than thealgorithm of D.D. Lee and H.S. Seung.
 
 Keywords - nonnegative matrix factorization; Krush-Kuhn-Tucker optimal condition; the stationarity point; updating anelement of matrix; updating matrices;
I.
 
I
NTRODUCTION
 Nonnegative matrix factorization approximation (NMF) isan approximate representation of a given nonnegative matrix
nxm
 R
by a product of two nonnegative matrices
nxr 
 R
and
rxm
 R H 
:
 H 
Because
is usually chosen by a very small number, thesize of the matrices
and
 H 
are much smaller than
. If 
is adata matrix of some object then
and
 H 
can be viewed as anapproximate representation of 
. Thus NMF can be consideredan effective technique for representing and reducing data.Although this technique appeared only recently, it has wideapplication, such as document clustering [7, 11], data mining[8], object recognition [5] and detecting forgery [10, 12]To measure the approximation in (1.1) often use theFrobenius norm of difference matrix
nim jijij
WH WH  H  f 
1122
))((2121),(
Thus NMF can be formulated as an optimization problemwith nonnegative constraints:
),(min
0,0
 H  f 
 H 
Since the objective function is not convex, most methodsfail to find the global optimal solution of the problem and onlyget the stationary point, i.e. the matrix pair
(W, H)
satisfies theKrush-Kuhn-Tucker (KKT) optimal condition [2]:
0,0
bjia
 H 
0))((,0))((
bjia
WH  H WH 

 jbaiWH  H   H WH 
bjbjiaia
,,,,0))(( ,0))((
Where
 H  f  H WH 
 / ),()(
 H  H  f WH 
 / ),()(
To update the
 H 
or
(the remaining fixed) often use thegradient direction reverse with a certain appropriate steps, sothat a reduction in the objective function, on the other still toensure non-negative of 
 H 
and
. Among the known algorithmssolve (1.3) must be mentioned algorithm LS (DD Lee and HSSeung [6]). This algorithm is a simple calculation scheme, easyto install and gives quite good results, so now it remains one of the algorithms are commonly used [10, 12]. LS algorithm isadjusted using the following formula:
ijijijijijijij
WH  A H   H  f  H  H 
)(~
ijijijijijijij
 H  H  H  A  f 
)~~~(~
  
1http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 10, No. 9, September 2012
By selecting
ij
and
ij
 
by the formula:
ijijij
WH  H 
)(

ijijij
 H  H 
)~~(
 

then the formula (1.5) and (1.6) become: 
ijijijij
WH  A H  H 
)()(~

ijijijij
 H  H  H  A
)~~()~(~

The adjustment formula uses multiplication so thisalgorithm is called the method of 
multiplicative update
. Withthis adjustment to ensure non-negative of 
~
and
 H 
~
. In [6]prove the monotonic decrease of the objective function afteradjustment:
),() ~,~(
 H  f  H  f 
This algorithm LS has the advantages of simple easy toimplement on the computer. However, the coefficients
ij
and
ij
 
are selected in a special way should not reach the minimumin each adjustment. This limits the speed of convergence of thealgorithm.To improve the convergence speed, E.F. Gonzalez and Y.Zhang has improved LS algorithm by using a coefficient foreach column of 
 H 
and a coefficient for each row of 
. In otherwords instead of (1.5) (1.6) using the following formula:
ijij jijij
WH  A H  H 
)(~
 
ijijiijij
 H  H  AH 
)~~(~
  
The coefficients
 j
 
and
i
 
are calculated through thefunction:
),,(
 xb Ag
 
(A is the matrix. B and x is the vector). This function isdefined as follows:
)(
 Axb Aq

q Ax A x p
) /(.

Where the symbol “./” and “
” denote component-wisedivision and multiplication, respectively. Then calculate
),,(
 xb Ag
 
by the formula:
    
0:max99.0,min
 p x Ap A pq p
   
The coefficients
 j
 
and
i
 
are determined by the function
g(A,b,x)
as follows:
n j H g
 j j j
..1),,,(
 
mi H g
iii
..1),,,(
 
However, the experiments showed that improvement of EFGonzalez and Y. Zhang has not really bring obvious effect.Also as remarks in [3], the LS algorithm and GZ algorithm (EFGonzalez - Y. Zhang [3]) are not guaranteed the convergenceto a stationary point. New algorithm uses addition for updating,so it is called additive update algorithm.In this paper we propose a new algorithm by updating eachelement of every matrix
and
 H 
based on the idea of nonlinear Gauss - Seidel method [4]. Also with someassumptions, the proposed algorithm ensures reachingstationary point (Theorem 2, subsection III.B). Experimentsshow that the proposed algorithm converges faster than thealgorithms LS and GZ.The content of the paper is organized as follows. In section 2,we present an algorithm to update an element of the matrix Wor H. This algorithm will be used in section 3 to construct anew algorithm for NMF (1.3). We also consider someconvergence properties of new algorithm. Section 4 presents ascheme for installing a new algorithm on a computer. Insection 5, we present experimental results comparing thecalculation speed of algorithms. Finally some conclusions aregiven in section 6.
 
II.
 
A
LGORITHM FOR UPDATING AN ELEMENT OF MATRIX
 A.
 
Updating an element of matrix W 
In this section, we consider the algorithm for updating anelement of 
, while retaining the remaining elements of 
and
 H 
. Suppose
ij
is adjusted by adding
 
parameter: 
 
ijij
~
If 
~
is an obtained matrix, then by some matrixoperations, we have:
mbia H WH  mbiaWH   H 
 jbibabab
..1,,)( ..1,,)( )~(
 
So from (1.2) it follows:
)(),(), ~(
 
g H  f  H  f 
2http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 10, No. 9, September 2012
Where
   
q pg
)(21)(
2
mb jb
 H  p
12
mb jbib
 H WH q
1
*)(
To minimize
),~(
 H  f 
, one needs to define
 
so that
)(
 
g
achieves the minimum value on the condition
0~
 
ijij
. Because
)(
 
g
is a quadratic function,then
 
can be defined as follows:
0,0),,max( 0,0
q pqqw pqq
ij
 
Formula (2.6) always means because if 
q
0
then by (2.4)we have
 p>0
From (2.3) and (2.6), we get:
0)(
 
g
, if (q = 0) or (q>0 and
ij
=0) (2.7.a)
0)(
 
g
, otherwise (2.7.b)By using update formulas (2.1) and (2.6), the monotonousdecrease of the objective function
 f(W,H)
is confirmed in thefollowing lemma .
 LEMMA 1: If conditions KTT are not satisfied at 
ij
 , then:
),(), ~(
 H  f  H  f 
Otherwise:
~
 Proof.
 
From (2.4), (2.5) it follows
ij
 H WH q
))((

Therefore, if conditions KTT (1.4) are not satisfied at
ij
,then properties
0,0,0
qq
ijij
cannot occur simultaneously. From this andbecause
0
ij
 ,
it follows that case (2.7.a) cannot happen. Socase (2.7.b) must occur and we have
0)(
 
g
. Therefore,from (2.2) we obtain
),(), ~(
 H  f  H  f 
Conversely, if (2.8) is satisfied, it means that:
q=0
or
q>0
 and
ij
= 0. So from (2.6), it follows
0
 
. Therefore, by(2.1) we have
ijij
~
Thus lemma is proved.
 B.
 
Updating an element of matrix H 
Let
 H 
~
be matrix obtained from the update rule:
 
ijij
 H  H 
~
where
 
is defined by the formulas:
naai
u
12
naajai
WH v
1
)(
0,0),,max( 0,0
vuvv H uvv
ij
 
By the same discussions used in lemma 1, we have
 LEMMA 2: If conditions KTT are not satisfied at H 
ij
 , then:
),() ~,(
 H  f  H  f 
Otherwise:
 H  H 
~
 
III.
 
T
HE
P
ROPOSED
A
LGORITHM
 A.
 
Updating matrices W and H 
In this section we consider the transformation
from
(W, H)
to (
~
,
 H 
~
) as follows:
 
Modify elements of 
by subsection II.A
3http://sites.google.com/site/ijcsis/ISSN 1947-5500

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->