1
`1-K-SVD: A Robust Dictionary Learning
Algorithm With Simultaneous Update
Subhadip Mukherjee, Rupam Basu, and Chandra Sekhar Seelamantula, Senior Member, IEEE
Abstract—We develop a dictionary learning algorithm by
minimizing the `1 distortion metric on the data term, which is
known to be robust for non-Gaussian noise contamination. The
proposed algorithm exploits the idea of iterative minimization of
arXiv:1410.0311v2 [cs.CV] 3 Mar 2015
weighted `2 error. We refer to this algorithm as `1 -K-SVD, where
the dictionary atoms and the corresponding sparse coefficients
are simultaneously updated to minimize the `1 objective, resulting
in noise-robustness. We demonstrate through experiments that
the `1 -K-SVD algorithm results in higher atom recovery rate
compared with the K-SVD and the robust dictionary learning
(RDL) algorithm proposed by Lu et al., both in Gaussian and
non-Gaussian noise conditions. We also show that, for fixed values
of sparsity, number of dictionary atoms, and data-dimension,
the `1 -K-SVD algorithm outperforms the K-SVD and RDL
algorithms when the training set available is small. We apply
the proposed algorithm for denoising natural images corrupted
by additive Gaussian and Laplacian noise. The images denoised
using `1 -K-SVD are observed to have slightly higher peak signal-
to-noise ratio (PSNR) over K-SVD for Laplacian noise, but the
improvement in structural similarity index (SSIM) is significant
(approximately 0.1) for lower values of input PSNR, indicating
the efficacy of the `1 metric.
Index Terms—Dictionary learning, sparsity, noise robustness,
`1 -minimization, iteratively re-weighted least-squares technique.
I. I NTRODUCTION
D ICTIONARY learning for sparse representation of sig-
nals has gained interest in the signal processing com-
munity in the past few years. A survey of the state-of-the-
art techniques for dictionary learning is given in [1], [2] and
references therein. Some seminal contributions to the problem
of data-adaptive dictionary learning were made by Aharon
et al. [3], [4], who proposed the K-SVD algorithm, which,
by far, is the most popular algorithm for dictionary design.
Theoretical guarantees on the performance of K-SVD can
be found in [5], [6]. Dai et al. [7] developed a generalized
framework for dictionary learning, known as SimCO, such that
the MOD [8] and K-SVD emerge as special cases. K-SVD
has been deployed to solve the problem of image denoising
[3], specifically to suppress zero-mean additive white Gaussian
noise. The principal idea behind the approach is to train
a dictionary that is capable of representing image patches
parsimoniously. Two training options have been proposed: (i)
training on a corpus of clean image patches; and (ii) training
on the noisy input image. Since the K-SVD algorithm is not
suitable for handling large image patches due to computational
overhead, addition of a global prior was proposed to enforce
The authors are with the Department of Electrical Engineering, Indian Insti-
tute of Science, Bangalore 560012, India. Emails: subhadip@ee.iisc.ernet.in,
rbasu28@ece.iisc.ernet.in, chandra.sekhar@ieee.org.
sparsity. The K-SVD uses the `2 distortion as a measure
of data fidelity. The dictionary learning problem has also
been addressed from the analysis perspective [9]–[11]. Apart
from denoising, dictionary-based techniques find applications
in image super-resolution, inpainting, etc. [12]–[14].
In many applications, the assumption of Gaussianness of
noise may not be accurate, thus rendering the `2 -minimization
based algorithms suboptimal. Moreover, algorithms based on
`2 -error minimization may lead to over-smoothing of the
image, causing loss of detail. One way to alleviate this problem
is to develop algorithms to minimize the `1 data error. We
develop a dictionary learning algorithm that approximates the
solution to the `1 minimization problem by iteratively solving
weighted `2 minimization problems, known as iteratively re-
weighted least squares (IRLS) [15]–[17]. We refer to the new
algorithm as `1 -K-SVD, which is a robust dictionary learning
algorithm that enables a simultaneous update of the dictionary
and the corresponding coefficients over the current support.
In dictionary learning for sparse coding, the standard met-
rics used for the data term and regularization are the `2
and `1 metrics, respectively. Algorithms for minimizing `1 -
based data terms have been proposed in the context of non-
negative matrix factorization (NMF) for document detection
[18]. Recently, the problem of robust dictionary learning
has been addressed by Lu et al. [19] in an online setting,
where the training examples are revealed sequentially [20].
Unlike K-SVD, their algorithm does not offer the flexibility
to simultaneously update the dictionary and the coefficients
over the currently estimated support.
Our contribution: We propose an algorithm for dictionary
learning aimed at minimizing the `1 distortion on the data
term. The `1 error is minimized using IRLS, using which we
solve the `1 minimization problem by iteratively solving a
series of `2 minimization problems (cf. Section II). The use of
`1 metric for data fidelity results in robustness for suppression
of impulsive noise from images while preserving structure,
as demonstrated by the experimental results (cf. Section III).
Unlike [19], in `1 -K-SVD, the dictionary atoms and the entries
of the coefficient matrix are simultaneously updated (similar
to K-SVD), offering better convergence performance.
II. P ROPOSED D ICTIONARY L EARNING A LGORITHM
Denote the training data by T , which contains N exemplars
N
{yn }n=1 in Rm , corrupted by additive noise. The objective
is to learn a dictionary D containing K atoms, tailored to
T , such that D represents the members of T using sparse
coefficient vectors xn . Typically, D is over-complete, meaning
 2

DISTANCE FROM TRUE DICTIONARY

that m < K. The symbols Y and X denote the matrices ! 1 -K-SVD
1

ATOM DETECTION RATE

0.2 K-SVD
constructed by stacking the training examples yn and the RDL 0.8

corresponding coefficients xn in the columns. In order to 0.15

0.6
achieve robustness to noise, we propose to solve the following 0.1
0.4
optimization problem: 0.05
0.2 ! 1 -K-SVD
N N K-SVD
0 RDL
X X 0
20 40 60 80 20 40 60 80
min kyn − Dxn k1 + λn kxn k1 . (1) ITERATIONS ITERATIONS
D,xn

DISTANCE FROM TRUE DICTIONARY

n=1 n=1 1
! 1 -K-SVD

ATOM DETECTION RATE

0.2
K-SVD
The cost function in (1) can be interpreted from a Bayesian RDL 0.8
0.15
perspective, where a Laplacian model is assumed for the 0.6
prior as well as for the additive noise. The parameters λn 0.1
0.4
in the regularization term adjust the sparsity of the resulting 0.05
0.2 ! 1 -K-SVD
representation versus data fidelity. To solve (1), we adopt the 0
K-SVD
RDL
0
alternating minimization strategy, wherein one starts with an 20 40
ITERATIONS
60 80 20 40
ITERATIONS
60 80

initial guess for D, and updates xn and D, alternately.

A. Sparse Coding
To update the coefficient vectors xn when D is fixed, one
needs to solve N independent problems of the form
FIG . 1: (Color online) Performance comparison of the `1 -K-SVD,
K-SVD, and RDL algorithms in terms of ADR and distance metric
κ, for Gaussian (first row) and Laplacian (second row) noise. The
plots are averaged over five independent realizations. The input SNR
is 20 dB. The number of examples in the training set is N = 1500.

DISTANCE FROM TRUE DICTIONARY

DISTANCE FROM TRUE DICTIONARY

DISTANCE FROM TRUE DICTIONARY

0.25 0.25
! 1 -K-SVD ! 1 -K-SVD
0.2
min yn − D̂xn + λn kxn k1 , (2) K-SVD K-SVD
0.2 RDL 0.2 RDL

xn 1 0.15 0.15
0.15

0.1
0.1 0.1
where D̂ denotes the estimate of the dictionary in the current 0.05 0.05
0.05
! 1 -K-SVD
K-SVD
iteration. To solve (2) using IRLS, one has to start with an 0
20 40 60 80
0
20 40 60 80
0 RDL
20 40 60 80
ITERATIONS ITERATIONS ITERATIONS
(0)
initial guess xn and update xn in the k th iteration as 1 1
! 1 -K-SVD 1 ! 1 -K-SVD
ATOM DETECTION RATE

ATOM DETECTION RATE

ATOM DETECTION RATE

K-SVD K-SVD
0.8 0.8 RDL 0.8 RDL
 −1
(k) (k) (k)
x(k+1)
n = D̂T W1n D̂ + λn W2n D̂T W1n yn , 0.6

0.4
0.6

0.4
0.6

0.4

0.2 ! 1 -K-SVD 0.2 0.2

where W1n and W2n are diagonal weight matrices of appropri- 0
K-SVD
RDL
0 0
20 40 60 80 20 40 60 80 20 40 60 80
ate dimensions. Typically, one begins the iterations by setting ITERATIONS ITERATIONS ITERATIONS

(0) (0)
W1n and W2n to identity matrices, for all n, and updating N = 600 N = 300 N = 200
th
in the k iteration as
FIG . 2: (Color online) Performance comparison of `1 -K-SVD, K-
(k+1) 1 (k+1) 1 SVD, and RDL algorithms for varying data-size N . The values of
W1n (j) =    , W2n (j) =   ,
 yn − D̂xn(k) +
 (k)  
 xn + m, s, and K are kept fixed at m = 20, s = 3, and K = 50. The
 j
  j

where W (j) denotes the j th diagonal entry of a diagonal
matrix W , and (x)j denotes the j th entry of the vector x.
A small positive constant  is added to the denominator to
ensure numerical stability. One can also choose to work with
an equivalent constrained version of (2) of the form
of dj and xj , one is required to solve
min
yn − D̂xn
subject to kxn k1 ≤ τn , (3)
xn 1
 
Ej − dj xj
s.t. S xj = Ωj and kdj k2 = 1,
where τn controls the sparsity of the resulting solution, with
smaller values of τn resulting in higher sparsity. Efficient
optimization packages, such as CVX [21] and ManOpt [22]
are available to solve problems of the form given in (2) and
(3). To further reduce the sensitivity of the solution to noise,
figures in the first and second rows show the variations of the distance
metric κ and the ADR, respectively, with iterations. The dataset is
corrupted by additive Laplacian noise with SNR 20 dB.
corresponding row in X, we focus on the error matrix Ej =
Y − i6=j di xi , resulting from the absence of dj , where xi
P
denotes the ith row of X. In order to obtain a refined estimate
min
(4)
j
dj ,x 1
where kAk1 denotes the sum of the absolute values of the
entries of matrix A. We seek to minimize

the cost function
subject to the constraint that S xj = Ωj , where S(·)
denotes the support operator, defined as S (z) = {i : zi 6= 0}.
we apply an appropriate threshold T0 on the entries of X, Therefore, it suffices to consider only those columns of Ej
following sparse coding. The optimum value of the threshold is whose indices are in the support set Ωj , denoted by Ej Ω .
j

application specific, and the values chosen in our experiments Furthermore, to get rid of scaling ambiguity, we restrict D
are indicated in Section III. Thus, the proposed approach may to have unit-length atoms, that is, kdi k2 = 1 for all i.
Consequently, solving (4) is equivalent to solving
also be interpreted as `1 minimization followed by pruning.
M
X
min E − uvT = min ken − vn uk1 s.t. kuk2 = 1, (5)

B. Dictionary Update u,v 1 u,v
n=1
Similar to the K-SVD algorithm, we adopt a sequential
K
updating strategy for the dictionary atoms {di }i=1 . To si- where en is the nth column of E and vn denotes the nth entry
th
multaneously update the j atom in the dictionary and the of the vector v. The key idea behind the `1 -K-SVD approach
 3

K-SVD `1 -K-SVD
σ = 15 σ = 25 σ = 35 σ = 15 σ = 25 σ = 35
Gaussian noise 31.72/31.23 28.66/28.26 26.43/26.18 30.77/30.08 28.58/28.00 25.98/26.22
0.80/0.81 0.68/0.67 0.56/0.54 0.78/0.77 0.73/0.73 0.62/0.61
Laplacian noise 31.50/31.04 28.39/27.98 26.12/25.86 31.70/30.96 28.46/28.30 26.79/26.55
0.79/0.80 0.67/0.66 0.55/0.53 0.81/0.80 0.76/0.75 0.66/0.60

TABLE II: (Color online) Comparison of output PSNR (DB) and SSIM (corresponding to the first and second rows in each cell, respectively)
for the K-SVD and `1 -K-SVD algorithms on the House (black) and Tower (red) images corrupted by additive Gaussian and Laplacian noise,
for different input noise levels. The values reported are obtained by averaging over five independent realizations. The values of PSNR and
SSIM did not change much from one realization to another.
Algorithm
1 (`1 -K-SVD): To find u and v such that found that J = 10 suffices for the convergence. The `1 -K-
E − uvT is minimized, subject to kuk = 1. SVD algorithm is computationally more expensive than the
1 2

1. Input: Error matrix E, number of iterations J. K-SVD and starts with the K-SVD initialization. However, the
2. Initialization: quality of the solution obtained using `1 -K-SVD is better than
(p) that obtained using K-SVD, in the case where the training set
• Set the iteration counter p ← 0, u ← a, and
(p) is contaminated by non-Gaussian as well as Gaussian noise.
v ← σb, where a and b are the left and right sin-
gular vectors, respectively, corresponding to the largest
III. E XPERIMENTAL R ESULTS
singular value σ of E.
(p) 1) Synthesized Signal: To validate the `1 -K-SVD algo-
• Initialize the weight matrices Wn ← I, for n =
1, 2, · · · , M , where I denotes the identity matrix and rithm, we first conduct experiments on synthesized training
M is the number of columns in E. data, for Gaussian as well as Laplacian noise. Since the
ground-truth dictionary D is known in the experiments, we
3. Iterate the following steps J times:
(p+1)
use two performance metrics, namely the atom detection rate
1
• wn (j) ← 

(p)
 
 en −vn u(p) + (ADR) and distance κ D̂, D of the estimated dictionary D̂
j
 
" M  2
#−1 " M
# from the ground-truth D. To compute ADR, we count the
number of recovered atoms in D. An atom di in D is consid-
X X
• u (p+1)
← vn(p) Wn(p+1) vn(p) Wn(p+1) en 
 T 
n=1
T
n=1 ered to be recovered if di d̂j  exceeds 0.99 for some j. Since
(p+1) (u(p+1) ) Wn en
• vn ← T , for n = 1, 2, · · · , M D and D̂ contain unit-length atoms, this criterion ensures a
(u(p+1)
) Wn (u(p+1) )
• p←p+1 near-accurate atom recovery. Finally, to compute ADR, we
u(p)
take the ratio of the number of recovered atoms to the total
4. Scaling: Set u ← and v ← u(p) 2 v(p)
ku(p) k2 number of atoms. The metric κ measures  of D̂
the closeness
 
5. Output: Solution (u, v) to (5). 1
PK
to D, and is defined as κ = K i=1 min 1 − dTi d̂j  .
 
1≤j≤K
The value of κ satisfies 0 < κ < 1. If κ is close to 0, it
σ = 15 σ = 25 σ = 35 indicates that the recovered dictionary closely matches with
λ = 1, 1 λ = 1, 1 λ = 8, 8 the ground-truth. The experimental setup is identical to that
np = 0.18, 0.15 np = 0.08, 0.05 np = 0.05, 0.04 in [4]. The ground-truth dictionary is created by randomly
generating K = 50 vectors in R20 , followed by column
TABLE I: (Color online) Values of the regularization parameter λ
normalization. Each column of the coefficient matrix X has
and the fraction of entries np retained in the thresholding step of
`1 -K-SVD, in case of Laplacian (black) and Gaussian (red) noise. s = 3 non-zeros, with their locations chosen uniformly at
random and amplitudes drawn from the N (0, 1) distribution.
is to approximate the `1 -norm using a reweighted `2 metric: Subsequently, D is combined with X and contaminated by ad-
M M ditive noise, to generate N = 1500 exemplars. All algorithms
are initialized with the training examples and the iterations are
X X T
ken − vn uk1 ≈ (en − vn u) Wn (en − vn u) .
n=1 n=1 repeated 80 times, as suggested in [4]. The optimum choice
of the hard-threshold T0 , found experimentally, is given by
Differentiating with respect to u and setting to zero, we get
T0 = 0.03kXkF . The variation of κ and ADR with iterations,
!−1 M
averaged over five independent trials, is demonstrated in Fig. 1.
M
!
X X
2
u= vn W n vn Wn en . To facilitate fair comparison of `1 -K-SVD with the K-SVD
n=1 n=1 and RDL [19], true values of the `1 and `0 norm of the ground-
T truth coefficient vectors, that is, true values of τn in (3) and the
Similarly, for a fixed u, we have vn = uuTW n en
Wn u
. Once the
th parameter s, respectively, are supplied to the algorithms. OMP
update of u and v is done, the j diagonal element of W
is used for sparse coding in K-SVD. The `1 -K-SVD algorithm,
should be updated as wn (j) ← (e −v1 u) + . Dictionary up-
| n n j|  as one can see from Fig. 1, results in a superior performance
date is performed by setting dj = u0 and xj Ωj = v0T , where over RDL and its `2 -based counterpart K-SVD, both in terms
(u0 , v0 ) solves (5). The step-by-step description to obtain the of κ and ADR. One can observe similar robustness of the
solution of (5) is given in Algorithm 1. Experimentally we `1 -K-SVD algorithm to both Gaussian and Laplacian noise
 4

from the experimental results. The faster convergence of the

`1 -K-SVD algorithm compared to RDL is attributed to the
simultaneous update of the dictionary and the coefficients.
However, they result in similar performance after sufficiently
many iterations. The execution times taken for each iteration
of the K-SVD, RDL and `1 -K-SVD are 0.53, 45.53, and 32.90
seconds, respectively, when executed on a MATLAB 2011
platform, running on a Macintosh OSX system with 8 GB
RAM and 3.2 GHz core-i5 processor.
To analyze the effect of the size of the training set on the (a) (b)
performance, we decrease the value of N , keeping m, s, and
K fixed. From the plots in Fig. 2, we observe that the `1 -
K-SVD algorithm is superior to RDL and K-SVD, when the
value of N is small. For N = 200, we note that the `1 -K-SVD
algorithm recovers 40% of the atoms accurately, whereas the
ADR for RDL and K-SVD is below 20%. This experiment
suggests that the `1 -K-SVD is particularly suitable when the
number of training examples is small.
2) Application to Image Denoising: We conduct image
denoising experiments for additive Gaussian and Laplacian (c) (d)
noise. The peak signal-to-noise ratio (PSNR) and structural
similarity index (SSIM) of the output images obtained using FIG . 3: Comparison of denoising performance of the K-SVD and `1 -
K-SVD algorithms for Laplacian noise: (a) Ground-truth clean image;
the K-SVD and `1 -K-SVD algorithms are compared. In our
(b) Noisy input, PSNR = 22.16 dB, SSIM = 0.36; (c) Denoised
experiments, dictionary learning is performed adaptively using image using K-SVD, PSNR = 29.78 dB, SSIM = 0.73; (d) Denoised
patches taken from the noisy image [3]. Patches of size 8 × 8 image using `1 -K-SVD, PSNR = 30.34 dB, SSIM = 0.81. The
are extracted from the noisy image, with an overlap of 4 pixels values of λ are np are 1 and 0.05, respectively.
in both directions. Thus, for an image of size 256 × 256, 3969
PSNR over K-SVD, but the improvement in SSIM is in the
noisy patches are extracted to train the dictionaries that are
range of 0.08 − 0.10. For visual comparison, we show the
chosen to be of size 64 × 128. Both K-SVD and the proposed
denoised output images obtained using K-SVD and `1 -K-SVD
`1 -K-SVD algorithms are initialized with the noisy patches,
in Fig. 3, where the input image is contaminated by Laplacian
and the iterations are repeated 10 times [3]. In the sparse cod-
noise, with PSNR of 22.16 dB. The values of PSNR and SSIM
ing stage of the K-SVD algorithm, we solve an optimization of the denoised images obtained using the two algorithms are
problem of the form min kxn k0 subject to yn − D̂xn ≤

xn 2 indicated in Fig. 3. We observe that the gain in PSNR using
1.15σ, where σ 2 is the variance of the noise and D̂ denotes `1 -K-SVD is slightly better than K-SVD, and the improvement
the current estimate of the dictionary. The factor 1.15 is fixed in SSIM is close to 0.1.
experimentally [3]. The `1 -K-SVD update of the coefficients is
obtained by solving (2) for each noisy patch yn , followed by IV. C ONCLUSIONS
hard-thresholding. The values of λn are chosen to be equal for We have developed a robust dictionary learning algorithm,
each patch, and denoted by λ. Hard-thresholding is performed referred to as the `1 -K-SVD, for minimizing the `1 error
by retaining a fraction np of K entries in each column of X. on the data term and applied it for image denoising. The
The values of λ and np depend on the type and strength of motivation behind using the `1 metric as a measure of data
the noise. As a rule of thumb, one must choose a larger value fidelity is to achieve robustness to non-Gaussian noise and to
of λ and smaller value for np as the value of σ increases. alleviate the problem of over-smoothing texture and edges in
The optimal values of these two parameters are obtained images. The `1 -K-SVD algorithm updates the dictionary and
experimentally and reported in Table I. The estimated clean the coefficients simultaneously, offering better flexibility and
patches are averaged to generate the overall denoised output. faster convergence compared to the RDL algorithm in [19].
The PSNR and SSIM of the denoised images, for additive Experiments on synthesized data indicate the superiority of
Laplacian and Gaussian noise of different levels σ, are reported the proposed algorithm over its `2 -based counterpart, namely
in Table II. We observe that the `1 -K-SVD algorithm yields the K-SVD algorithm, in terms of ADR and the distance of
an output PSNR comparable with the K-SVD, and results in the recovered dictionary to the ground-truth. We have also
a higher value of SSIM over K-SVD, especially when σ is demonstrated the superiority of the `1 metric for the case
large. For images corrupted by Gaussian noise with low value where the number of training examples is limited. Image
of σ, K-SVD results in a better denoising performance than denoising experiments showed that the `1 -K-SVD algorithm
`1 -K-SVD. However, as the noise strength σ increases, `1 - results in output images with slightly higher PSNR and SSIM
K-SVD does a better job of preserving the image structure, values, at low input PSNRs, compared with the K-SVD
as reflected in the higher SSIM values. When the image is algorithm. The `1 -K-SVD algorithm may also turn out to be
corrupted by Laplacian noise, `1 -K-SVD yields slightly better useful in other applications such as image super-resolution.
 5

R EFERENCES
[1] R. Rubinstein, A. M. Bruckstein, and M. Elad, “Dictionaries for sparse
representation modeling,” Proceedings of the IEEE, vol. 98, no. 6, pp.
1045–1057, Jun. 2010.
[2] I. Tosic and P. Frossard, “Dictionary learning,” IEEE Sig. Proc. Mag.,
vol. 28, no. 2, pp. 27–38, Mar. 2011.
[3] M. Elad and M. Aharon, “Image denoising via sparse and redundant
representations over learned dictionaries,” IEEE Trans. Image Proc., vol.
15, no. 12, pp. 3736–3745, Dec. 2006.
[4] M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for
designing overcomplete dictionaries for sparse representation,” IEEE
Trans. Sig. Proc., vol. 54, no. 11, pp. 4311–4322, Nov. 2006.
[5] S. Arora, R. Ge, and A. Moitra, “New algorithms for learning incoherent
and overcomplete dictionaries,” arXiv:1308.6273v5, May 2014.
[6] K. Schnass, “On the identifiability of overcomplete dictionaries via the
minimisation principle underlying K-SVD,” arXiv:1301.3375v4, Mar.
2013.
[7] W. Dai, T. Xu, and W. Wang, “Simultaneous codeword optimization
(SimCO) for dictionary update and learning,” IEEE Trans. on Sig. Proc.,
vol. 60, no. 12, pp. 6340–6353, Dec. 2012.
[8] K. Engan, S. O. Aase, and J. H. Husoy, “Method of optimal directions
for frame design,” in Proc. IEEE Intl. Conf. on Accoust., Speech, and
Sig. Proc., pp. 2443–2446, Mar. 1999.
[9] R. Rubinstein, T. Peleg, and M. Elad “Analysis K-SVD: A dictionary-
learning algorithm for the analysis sparse model,” IEEE Trans. Sig.
Proc., vol. 61, no. 3, pp. 661–677, Feb. 2013.
[10] E. M. Eksioglu and O. Bayir, “K-SVD meets transform learning:
Transform K-SVD,” IEEE Sig. Proc. Lett., vol. 21, no. 3, pp. 347–351,
Mar. 2014.
[11] J. Dong, W. Wang, and W. Dai, “Analysis SimCO: A new algorithm
for analysis dictionary learning,” in Proc. IEEE Intl. Conf. on Accoust.,
Speech, and Sig. Proc., pp. 7193–7197, May 2014.
[12] J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via
sparse representation,” IEEE Trans. Image Proc., vol. 19, no. 11, pp.
2861–2873, Nov. 2010.
[13] S. Sahoo and W. Lu, “Image in-painting using sparse approximation
with adaptive window selection,” in Proc. IEEE Intl. Symposium on
Intelligent Sig. Proc., pp. 1–4, Sep. 2011.
[14] J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image
restoration, IEEE Trans. Image Proc., vol. 17, pp. 53–69, Jan. 2008.
[15] I. Daubechies, R. DeVore, M. Fornasier, and C. S. Güntürk, “Iteratively
re-weighted least-squares minimization for sparse recovery,” Communi-
cations on Pure and Applied Math., vol. LXIII, pp. 1–38, 2010.
[16] C. J. Miosso, R. V. Borries, M. Argáez, L. Velazquez, C. Quintero, and
C. M. Potes, “Compressive sensing reconstruction with prior information
by iteratively reweighted least-squares,” IEEE Trans. on Sig. Proc., vol.
57, no. 6, pp. 2424–2431, Jun. 2009.
[17] O. Endra and Gunawan, “Comparison of `1 -minimization and iteratively
reweighted least squares-`p -minimization for image reconstruction from
compressive sensing,” in Proc. Second Intl. Conf. on Advances in
Computing, Control and Telecom. Tech., pp. 85–88, Dec. 2010.
[18] S. P. Kasiviswanathan, H. Wang, A. Banerjee, and P. Melville, “Online
`1 dictionary learning with application to novel document detection,”
Advances in Neural Info. Proc. Systems, vol. 25, pp. 2267–2275, 2012.
[19] C. Lu, J. Shi, and J. Jia, “Online robust dictionary learning,” in Proc.
IEEE Conf. on Comp. Vis. and Patt. Recog., pp. 415–422, 2013.
[20] J. Mairal, F. Bach, J. Ponce, and G. Sapiro, “Online learning for matrix
factorization and sparse coding,” J. of Mach. Learning Res., vol. 11, pp.
19–60, 2010.
[21] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex
programming,” version 2.0 beta, Sep. 2013.
[22] N. Boumal, B. Mishra, P. A. Absil, and R. Sepulchre, “ManOpt, a
MATLAB toolbox for optimization on manifolds,” J. of Mach. Learning
Res., vol. 15, pp. 1455–1459, 2014.