Professional Documents
Culture Documents
独立成分分析の基礎と応用
独立成分分析の基礎と応用
(Hiromu Gotanda)
(Takaaki Ishibashi)
(Nobuo Iwasaki)
(Katsuhiro Inoue)
()
(BSS:
Blind Source Separation) BSS
BSS
(ICA: Independent Component Analysis) [1, 2, 3],
(TFM: Time Frequency Masking) [4, 5], [6],
[7], [8]
ICA
ICA ICA
(NG: Natural Gradient) [9] FastICA [10]
FastICA
ICA
(ICA)
ICA
7
2.1
$N(\geq 2)$
(1)
$x(t)=As(t)$
()
()
$(N\cross N),$
$a_{mn}$
$n$
$N$
$A$
$T$
$m$
$a_{mn}$
(2)
$u(t)=Wx(t)$
$u(t)=[u_{1}(t), u_{2}(t), \cdots , u_{N}(t)]^{T}$
$W$
$(N\cross N)$
$w_{nm}$
$\{x(t)|t=1,2, \cdots\}$
$W$ $\{u_{n}(t)|t=1,2, \cdots\}$
ICA
2.2
KL
$s$
$u$
$p(s_{n})$
2.
(3)
(2) $W$
(3) $p(u)$ $q(u)$
KL (Kullback-Leibler)
$q(u)$
$\int p(u)\log\frac{p(u)}{q(u)}du$
$\Pi_{n=1}^{N}p(u_{n})$
(4)
KL
$s(t)$ ,
$s$
8
23
KL W
KL $W$
KL $W$
(4) KL
$KL(u)$
$=$
(5)
$\sum_{n=1}^{N}\mathcal{H}(u_{n})-\mathcal{H}(u)$
(2)
$p(x)=p(u)/|W|$
$|\cdot|$
KL $W$
$\mathcal{H}(u)=\mathcal{H}(x)+\log|W|$
$KL(W)$
$=$
(6)
$\sum_{n=1}^{N}\mathcal{H}(u_{n})-\log|W|-\mathcal{H}(x)$
2.4
$KL(W)$
()
$W$
(7)
2 3.
()
$I$
$\eta$
$\varphi(u_{n})$
$W$
Amari [9],
$W$
(NG: Natural Gradient)
$N^{2}$
$w_{nm}$
$w_{nm}$
(8)
2.5
ICA
ICA InfoMax
[12]. $\mathcal{I}(u;x)=\mathcal{H}(u)-\mathcal{H}(u|x))$
(5)
$\mathcal{I}(u;x)=\Sigma_{n=1}^{N}\mathcal{H}(u_{n})-\mathcal{H}(u)$
$)$
$\varphi(u)\approx-d\log p(u)/du$
$p(u)$
$\varphi(u)=\tanh(u)$ ,
$\varphi(u)=u^{3}$
9
$\mathcal{H}(u)=\mathcal{H}(x)+\log|W|$ InfoMax
InfoMax
Extended-InfoMax [13].
$l(W)=\Sigma_{n=1}^{N}\ln\{|W|\Pi_{n=1}^{N}r(w_{n}^{T}x)\}$ ICA
[14]. $W$
(4) KL
$w_{n}$
$r(\cdot)$
$n$
$p(\cdot)$
$r(\cdot)$
$r(\cdot)$
$W$
$W$
$KL(W)\approx-\ell(W)$
InfoMax ICA
(8)
$p(\cdot)$
FastICA
$KL(W)= \sum_{n=1}^{N}\mathcal{H}(u_{n})-\log|W|-\mathcal{H}(x)$
KL (6),
$KL(W)$
4, $KL(W)$ $W$ 1 ()
$W$ $|W|=1$ $(W^{T}W=I)$
$\mathcal{H}(x)$
KL
$KL(W)$
$\approx$
(9)
$\sum_{n=1}^{N}\mathcal{H}(u_{n})$
KL
$u_{n}$
Hyv\"arinen
ICA FastICA [10] [11].
3.1
$W$
$W$
()
$\Gamma=[c_{1},
\cdots,
c_{n}]$
diag
1 5,
$\acute{x}=x-E[x]$
$E[\acute{x}\acute{x}^{T}]$
$\Lambda=$
$\lambda_{n}$
$c_{n}$
$\tilde{x}=\Lambda^{-1/2}\Gamma^{T_{X}^{\ovalbox{\tt\small REJECT}}}$
$E[\tilde{x}\tilde{x}^{T}]=I$
$\tilde{x}$
$s_{m}$
$s$
(10)
$\tilde{x}=\Lambda^{-1/2}\Gamma^{T}As$
$N^{2}$
10
$\tilde{A}\tilde{A}^{T}=I$
(2) (1)
$\tilde{A}=\Lambda^{-1/2}\Gamma^{T}A$
$W$
$\Vert w_{n}\Vert^{2}=1$
6, (ICA )
$W$
3.2
FastICA
(9)
(11)
$u_{n}=w_{n}^{\tau_{\tilde{X}}}$
$w_{n}$
$\mathcal{H}(u_{n})$
$W$
$\mathcal{H}(u_{n})$
Hyv\"arinen
$\mathcal{H}(u_{n})$
[2],
(12)
$J(u_{n})=\mathcal{H}(\nu)-\mathcal{H}(u_{n})\geq 0$
1 7.
$0$
$\nu$
$J(u_{n})\geq 0$
$u_{n}$
$0$
$u_{n}$
$J(u_{n})$
$\Vert w_{n}\Vert^{2}=1$
(13)
$L(w_{n})=\{E[G(u_{n})]-E[G(\nu)]\}^{2}-\beta\{\Vert w_{n}\Vert^{2}-1\}$
2
(13)
$G(\cdot)$
$\beta$
$)$
(14)
$E[\tilde{x}g(w_{n}^{T}\tilde{x})]-\beta w_{n}=0$
$g(\cdot)$
$\varphi(\cdot)$
$G(\cdot)$
(8)
$w_{n}^{+}$
$=$
$w_{n}$
$=$
$w_{n}$
$E[\tilde{x}g(w_{n}^{T}\tilde{x})]-E[g(w_{n}^{T}\tilde{x})]w_{n}$
(15)
(16)
$\frac{w_{n}^{+}}{\Vert w_{n}+\Vert}$
6 2 2
$w_{n}$
7
$\nu$
$u_{n}$
FastICA
1
$=\Vert w_{n}^{T}w_{n}\Vert=1$
$\nu$
$E[u_{n}^{2}]=E[w_{n}^{T}\tilde{x}\overline{x}^{T}w_{n}]=w_{n}^{T}E[\tilde{x}\tilde{x}^{T}]w_{n}$
$u_{n}$
$0$
11
FastICA
(15) (16)
(17)
$|w_{n,01d}^{T}w_{n,new}|\simeq 1$
old new
$(n=1)$ #
$N$
9.
$n=2$
$w_{n}$
$u_{1}=w_{1}^{T_{\tilde{X}}}$
$w_{1}$
$s_{m}$
$w_{2}$
$n\geq 2$
$w_{n}$
$w_{i}(i\leq
n-1)$
(18)
$w_{n}=w_{n}- \sum_{i=1}^{n-1}w_{i}^{T}w_{n}w_{n}$
(16)
1 (DeflaFastICA
$w_{n}$
3.3
FastICA ICA
FastICA InfoMax KL
FastICA
(4) $p(u_{1}, \cdots , u_{N})$
2 (KL ) (9) $p(u_{n})$
() FastICA
$W=[w_{1}, w_{2}, \cdots , w_{N}]^{T}$
$\prod_{n=1}^{N}p(u_{n})$
()
$x_{n}$
(
)
()
FastICA
$w_{n}$
(12)
9 4 (1) ICA
$u_{1}$
$s_{1}$
12
$W$
FastICA
(13) 10.
$w_{n}$
FastICA
$)$
FastICA
(14) (15) (16)
$\eta$
$\eta$
(17)
$\beta$
2 (1)
(19)
$x_{m}(t)= \sum_{n=1}^{N}a_{mn}(t)*s_{n}(t)=\sum_{n=1}^{N}\sum_{t=0}^{T-1}a_{mn}(t)s_{n}(t-t)$
$s_{n}(t)(n=1,2,
\cdots, N)$
$\{x_{m}(t)|t=0,1,2, \cdots\}$
$m$ (),
$a_{mn}(t)$
$n$
$T$
(19)
$t$
$m$
4.1
$*$
ICA
$u_{m}(t)= \sum_{m=1}^{N}w_{nm}(t)*x_{m}(t)=\sum_{m=1}^{N}\sum_{t=0}^{T-1}w_{nm}(t)x_{m}(t-t)$
$T$
$w_{nm}(t)$
(20)
TDICA $T_{60}[\sec]11$
$T_{60}[\sec]$
$W$
10 InfoMax
11 $60dB$
13
$8KHz$
$T_{60}=50[m\sec]$
$T=400$ 12,
$T=T$ 1 400
$\{w_{nm}(t)|t=0,1,2, \cdots, T-1\}$
13.
4.2
ICA
(19)
(21)
$\omega_{l}$
$k$
$l$
$a_{mn}(t)$
$A(\omega_{l})$
$a_{mn}(\omega_{l})$
$s_{n}(\omega_{l}, k)$
$x_{m}(\omega_{l}, k)$
()
$\{x_{m}(l+kR)|l=$
$\{x_{m}(t)|t=0,1,2, \cdots\}$
$L$
$0,1,2,$
$\cdots,$
$L-1\}$
$k$
$l=0,1,2,$
$\cdots,$
(22)
$L-1$
$R$
$j=\sqrt{-1}$
(), $h(l)$
(21) (20)
$x_{m}(\omega_{l}, k)$
L-1)$
(23)
),
$\{x_{m}(\omega_{l},
k)|k=0,1,2, \cdots, K\}$
$W(\omega_{l})$
$\omega_{l}$
$W(\omega_{l})$
$u(\omega_{l}, k)$
$W(\omega_{l})$
$K\}$
12
[15].
$T$
$T_{60}[\sec]$
$f_{s}$
[Hz]
$T\approx T_{60}f_{s}$
13
14
(NG) (8)
$W(\omega_{l})arrow W(\omega_{l})-\eta E[\varphi(u(\omega_{l}, k))u(\omega_{l}, k)^{H}-$
[16].
$\Re$
$\Im$
FastICA (15) (16)
$H$
$w_{n}^{+}(\omega_{l}, k)$
$arrow$
(25)
$w_{n}(\omega_{l}, k)$
$arrow$
[17].
(26)
$\frac{w_{n}^{+}(\omega_{l},k)}{\Vert w_{n}^{+}(\omega_{l},k)\Vert}$
$\tilde{x}(\omega_{l}, k)$
$\omega\iota$
,,
(25)
(24)
$g(\cdot)$
$g(\cdot)$
$\varphi(\cdot)$
FastICA
CPU
NG
43
(21)
$\iota$
$W(\omega_{l})$
(27)
$W(\omega_{l})A(\omega_{l})=P(\omega_{l})D(\omega_{l})$
$D(\omega_{l})$
$W(\omega_{l})$
$W(\omega_{l})x(\omega_{l}, k)$
$n$
$i(\neq n)$
$n$
$i$
$n$
$d_{i}(\omega_{l})$
$d_{i}(\omega_{l})$
$l$
(23)
$u_{n}(t)$
43.1
$s_{n}(t)$
$\omega_{l}$
$k$
14 1
15
$\sim$
$\tilde{n}$
$u_{\tilde{n}}$
$N$
$u$
$u_{\overline{n}}$
$u$
],
Murata [18]
$\xi_{\tilde{n}}$
$nl$
11
(28)
$=$
(Decomposed Spectrum; )
$n$
[19] [20].
$s_{n}$
$\xi_{m\overline{n}}=a_{mn}s_{n}$
$u_{\tilde{n}}$
$n$
$s_{n}$
$\tilde{n}$
$m=1,2,$
$\cdots,$
$u$
$\xi_{\overline{n}}$
$m$
$\xi_{m\tilde{n}}$
(29)
$N$
$\tilde{n}$
$\tilde{n}$
$s_{n}$
$u_{\tilde{n}}$
$u_{\tilde{n}}$
(28)
$\tilde{n}$
$\xi_{\tilde{n}}$
(29)
$\xi_{\tilde{n}}$
[20].
$n$
$\tilde{n}$
$\xi_{m\tilde{n}}$
$s_{n}$
2 1 $m$
1
1
$\xi_{m\tilde{n}}$
$a_{mn}$
$\tilde{n}$
$n$
$\xi_{m\tilde{n}}$
$m$
$a_{mn}$
$m$
[ 3
$(a_{mn})$
$s_{n}$
2 3
$n$
$u_{\tilde{n}}$
() (
ICA
$)$
16
432
(29)
$\omega_{l}$
$k$
$m=1,2,$
$\cdots,$
(30)
$N$
$a_{mn}(\omega_{l})$
$\xi_{m\overline{n}}(\omega_{l}, k)$
$n$
$\xi_{m\tilde{n}}(\omega_{l}$
(
[19] [21].
(30)
2 2
$n=2$
$n=1$ $m=1$
$m=2$
$a_{mn}(\omega_{l})$
$m$
$\omega_{l}$
$)$
$\xi_{m\tilde{n}}(\omega_{l}, k))$
$a_{mn}(\omega_{l})$
$|a_{nn}(\omega_{l})|>|a_{mn}(\omega_{l})|,$
for
(31)
$m\neq n$
(30)
$|\xi_{11}(\omega_{l}, k)|>|\xi_{21}(\omega_{l}, k)|,$
(32)
(33)
[19] [22].
$\xi_{\tilde{n}}$
(32)
(33)
[22].
[23].
2
$\theta_{2}(\omega_{l})$
$s_{1}(t)$
$s_{2}(t)$
$W(\omega_{l})$
$\theta_{1}(\omega_{l})$
$\hat{\theta}_{1}(\omega_{l})$
$=$
$\cos^{-1}(\frac{c(\angle[W^{-1}(\omega_{l})]_{21}-\angle[W^{-1}(\omega_{l})]_{11})}{2dF_{s}\omega_{l}})$
(34)
$\hat{\theta}_{2}(\omega_{l})$
$=$
$\cos^{-1}(\frac{c(\angle[W^{-1}(\omega_{l})]_{22}-\angle[W^{-1}(\omega_{l})]_{12})}{2dF_{s}\omega_{l}})$
(35)
(30)
$c$
$[W^{-1}(\omega_{l})]_{nm}$
$W(\omega_{l})$
$\hat{\theta}_{1}(\omega_{l})$
$\hat{\theta}_{2}(\omega_{l})$
$d$
$W(\omega_{l})^{-1}$
$(n, m)$
17
2
(30) $N=50$
[20]. LAN
(OFDM: Orthogonal Frequency Division Multiplexing)
()
[24]. OFDM
$|\xi_{n\tilde{n}}|=|a_{nn}s_{n}|>|a_{mn}s_{n}|=|\xi_{m\tilde{n}}|$
$f$ $or$
(36)
$m\neq n$
(36)
$m$ $m=n$
$a_{mn}$
$s_{n}$
$\xi_{m\overline{n}}$
$n$
$m$
$n$
$\xi_{m\tilde{n}}$
(ICA) FastICA
Infomax
ICA FastICA
ICA(TDICA)
ICA(FDICA) 2
ICA
$+$ [msec]
1
(21)
(21)
[25]
ICA
ICA
[26][27],
[1] A. Cichocki and S. Amari: Adaptive blind signal and image processing, learning
algorithm and applications; John Wiley & $Sons$ (2002)
[2] A. Hyv\"arinen, J. Karhunen and E. Oja: Independent component analysis; John Wiley
& $Sons$ (2001)
18
[4]
. Yilmaz and
[5] K. Fujita: Remarks on a method of blind source separation; Information, Vol. 13,
No. 3(B), pp. 829-834 (2010)
[6] B. A. Olshausen and D. J. Field: Sparse coding of sensory inputs; Current Opinion
in Neurobiology, Vol. 14, pp. 481-487 (2004)
[7] A. Cichocki, R. Zdunek and S. Amari: Nonnegative matrix and tensor factorization;
IEEE Signal Processing Magazine Vol. 25, No. 1, pp. 142-145 (2008)
[10] A. Hyv\"arinen and E. Oja: Independent component analysis: algorithms and applications; Neuml Networks, Vol. 13, No. 4-5, pp. 411-430 (2000)
[11] A. Hyv\"arinen: Fast and Robust Fixed-Point Algorithms for Independent Component
Analysis; IEEE Trans. Neuml Networks, Vol. 10, No. 3, pp. 626-634 (1999)
[12] A. J. Bell and T. J. Sejnowski: An information maximization approach to blind
separation and blind deconvolution; Neuml Computation, Vol. 7, No. 6, pp. 11291159 (1995)
[13] T. W. Lee, M. Girolami and T. J. Sejnowski: Independent Component Analysis
Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian
Sources; Neural Computation, Vol. 11, No. 2, pp.417-441 (1999)
[18] N. Murata, S. Ikeda and A. Ziehe: An approach to blind source separation based
on temporal structure of speech signals; Neurocomputing, Vol. 41, Issue 1-4, pp. 1-24
(2001)
19
[19] H. Gotanda, K. Nobu, T. Koya, K. Kaneda, T. Ishibashi, N. Haratani: Permutation
correction and speech extraction based on split spectrum through FastICA; $Pmc$ .
ICA2003, pp. 379-384 (2003)
[20] :ICA OFDM
; A, Vol. J91-A, No. 4, pp. 448-457 (2008)
[21] K. Nobu, T. Koya, K. Kaneda, N. Haratani and H. Gotanda: Noise Reduction Using
Locational Information on Target Sound Source; J. Robotics and Mechatronics, Vol.
15, No. 1, pp. 15-23 (2003).
[22] :
ICA ; Vol. 19, No. 12, $pp$ .
471-478 (2006)
[23] :
; A, Vol. J87-A, No. 7, pp. 1065-1069 (2004)
[24] :
QAM-OFDM ;
, Vol. J92-A, No. 3, pp. 141-149 (2009)
$A$
[25] :
; Vol. 22, No. 8, pp. 287-294 (2009)
[26] J. Karvanen and V. Koivunen: Blind separation methods based on Pearson system
and its extensions; Signal Pmcessing, Vol. 82, No. 4, pp. 663-673 (2002)
[27] F. R. Bach and M.I. Jordan: Kernel independent component analysis; J. Machine
Leaming Research, Vol. 3, pp. 1-48 (2002)
ICA
$s_{n}$
$u_{\tilde{n}}$
$\xi_{\overline{n}}$
$m$
$\xi_{m\overline{n}}$
$\xi_{m\tilde{n}}=a_{mn}s_{n}$
$m=1,2$ ,
$u_{\tilde{n}}$
$s_{n}$
$\cdot\cdot\cdot$
(37)
$N$
$N$
ICA
$N$
$x=$
$u=PDs$
(38)
20
diag $[d_{1}, d_{2}, \cdots, d_{N}]$
1
$D=$
$P$
$A$
$W$
(39)
$u=Wx=WAs$
(39)
(40)
$\{\begin{array}{l}u_{\tilde{1}}00|0\end{array}\}+\{\begin{array}{l}0u_{\tilde{2}}0|0\end{array}\}+\cdots+\{\begin{array}{l}000|u_{\tilde{N}}\end{array}\}=W\sum_{n=1}^{N}\{\begin{array}{l}a_{1n}a_{2n}|a_{N,n}\end{array}\}s_{n}$
$\tilde{n}$
$u_{\tilde{n}}$
$n$
$s_{n}$
(41)
$u_{\tilde{n}}=d_{n}s_{n}$
$(s_{n}=0)$ (40)
$n$
$s_{n}$
$(s_{n}\neq 0)$
$s_{n}$
$[u_{0}000^{\tilde{n}}$
$\tilde{n}$
$u_{\overline{n}}$
(42)
$=W\{\begin{array}{l}a_{1n}a_{2n}\vdots a_{Nn}\end{array}\}s_{n}$
(43)
(42)
(44)
$\{\begin{array}{l}\xi_{1\tilde{n}}\xi_{2\tilde{n}}\vdots\xi_{N\tilde{n}}\end{array}\}=\{\begin{array}{l}a_{1n}a_{2n}\vdots a_{N,n}\end{array}\}s_{n}$
$\xi_{m\overline{n}}=a_{mn^{S}n}$
$m=1,2,$
$\cdots,$
(45)
$N$
(37) (45)
$(m=1,2, \cdots, N)$
$a_{mn}$
$s_{n}$
$\xi_{\overline{n}}$
$\xi_{m\overline{n}}$
$m$
$N$
$m$
$a_{mn}$