## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

**Kernel Fisher Discriminant for Texture Classiﬁcation
**

Jianguo Zhang and Kai-Kuang Ma † , Senior Member, IEEE School of Electrical and Electronic Engineering Nanyang Technological University, Singapore Nanyang Avenue, 639798

† Email:

ekkma@ntu.edu.sg

Abstract Kernel Fisher discriminant (KFD) is a state-of-the-art nonlinear machine learning method, and it has great potential to outperform linear Fisher discriminant. In this paper, a nonlinear discriminative texture feature extraction method based on KFD is proposed for texture classiﬁcation. We mathematically show that ﬁnding the optimal discriminative texture features is equivalent to ﬁnding the optimal discriminative projection directions of the input data by KFD. Our proposed KFD-based method integrates texture feature extraction, nonlinear dimensionality reduction, and discrimination in a uniﬁed framework. Optimized and closed-form solutions are derived for both two-class and multi-class texture classiﬁcation problems, individually. Extensive experimental results clearly show that the proposed method yields excellent performance in texture classiﬁcation and outperforms other texture recognition methods based on discriminative classiﬁers.

I. I NTRODUCTION Texture analysis plays an instrumental role in computer vision and image processing. It has received considerable amount of attentions in the past few decades, and numerous methods have been proposed for texture feature extraction and classiﬁcation. Extensive applications of texture analysis involve medical imaging, remote sensing, industrial inspection, document segmentation, biometrics, texture-based image retrieval, and so on. Existing texture analysis methods can be broadly classiﬁed into four categories [1]: statistical methods, model-based methods, signal

September 7, 2004

DRAFT

2

processing methods, and structural methods. Among them, the methods from the ﬁrst three are commonly encounterd and will be reviewed as follows. From the purpose of feature extraction, texture features can be broadly divided into two categories: 1) texture features for representation; 2) texture features for discrimination. In the past, most of the texture features are extracted based on representation. A comparative study conducted by Randen et al. [2] suggests that it is more preferable to extract texture features based on discrimination instead of representation for texture classiﬁcation. Hence learning discriminative texture features for texture classiﬁcation is desirable. Statistical methods are motivated from Julesz’s ﬁndings that human visual systems usually recognize textured objects based on the statistical distribution of their image gray levels via the ﬁrst-order, the second-order, or the higher-order statistics [3][4][5]. The most commonly used and referenced method is the so-called gray-level co-occurrence matrix (GLCM) pioneered by Haralick [6], which estimates texture properties related to the second-order statistics. It is worth pointing out that although statistical texture features are usually used to classify texture regions, most of them are extracted explicitly or implicitly based on statistical texture representation. For model-based methods, textures are often viewed as mathematical image perceptual models. The key problems of these methods are how to choose a suitable model for characterizing the selected textures and how to estimate the parameters of these models based on some criteria. Another concern is that intensive computations are usually required to determine the proper parameters. The derived model parameters are used as the features to capture the perceived essential qualities of texture. Commonly-used texture models are autoregressive model (AR) [2], Markov random ﬁeld (MRF) [7], and Wold decomposition model [8]. AR model is a linear model established from the training samples through the least-mean-squares ﬁtting. MRF attempts to describe the relationship of texture pixels by the deﬁnition of cliques within a region of interest. Its optimization is based on maximizing the posteriori probability. Wold decomposition model is a perceptual texture model that decomposes textures into deterministic and non-deterministic ﬁelds, which correspond to regular texture component and random texture component, respectively. Usually, these models can capture the local contextual information in a textured image. However, the model parameters are optimized based on image representation instead of image discrimination. Signal processing methods provide another set of powerful approaches for texture analysis, such as multichannel Gabor ﬁlter, wavelet transform, ﬁnite impulse response (FIR) ﬁlter, etc. Gabor ﬁlter is appealing because of its simplicity and support from neurophysiological experiments [9].

September 7, 2004

DRAFT

In this paper. which gives rise to the ﬁlter selection problem. Accordingly. dimensionality reduction and discrimination together. our proposed approach is to extract features that could maximize the separation or discrimination among different textures. textures can be classiﬁed by only a small set of ﬁlters. However. texture images are usually decomposed into several feature images through projection by using a set of selected ﬁlters. Moreover.3 Jain et al. the performs feature extraction. Compared to multichannel texture ﬁlters such as Gabor or wavelet ﬁlter. a kernel version of Fisher discriminant is recently developed for the nonlinear discriminative feature extraction. However its linear optimal solution heavily depends on the assumption that the input patterns have equal covariance matrix. a nonlinear texture feature extraction method based on KFD is proposed. To overcome this limitation. Hence. On the other hand. Jain et al. the proposed method is optimized based on discriminant instead of texture representation. employ neural network to select a minimum set of Gabor ﬁlters for texture discrimination while keeping the performance at an acceptable level compared to the case without ﬁlters selection [11]. For a c-class problem. The optimal solution of kernel Fisher discriminant (KFD) corresponds to the optimal Bayesian classiﬁer which accounts for the minimization of the classiﬁcation (Bayesian) error rate [12] [13]. For two-class problem. This assumption is usually not true for real-world data. extending KFD for multi-class texture classiﬁcation is desirable. In this paper. In these ﬁltering methods. use Gabor ﬁlters for texture segmentation despite that it is designed based on texture reconstruction [10]. Unlike AR texture features. and classiﬁcation simultaneously. the proposed KFD-based texture classiﬁcation method optimizes the texture discrimination mask and September 7. KFD is originally proposed for two-class problem [13]. KFD performs nonlinear feature dimensionality reduction based on discrimination. For this problem. These ﬁlters are often designed based on representation such that textures could be reconstructed with minimum information loss. nonlinear dimensionality reduction. a texture classiﬁcation method is also described. we generalize it from two classes to multiple classes to deal with multi-class texture classiﬁcation problem (denoted as multi-class KFD). In texture ﬁltering methods. 2004 DRAFT . a general ﬁlter bank is often too large because it is designed to capture general texture properties. KFD simultaneously ﬁnds out c − 1 optimal projection directions resulting in c − 1 dimensional texture features via one-shot optimization. linear Fisher discriminant (LFD) is a well-known linear discrimination technique which can incorporate feature extraction. For the purpose of pattern discrimination. Unlike other texture feature extraction methods. KFD ﬁnds only one projection direction followed by one-dimensional texture features.

xj (m − R + 1. C − 1) ω(1. n) by rows. ω(0. and a rectangular region of support is the most commonly used one. n) . September 7. ω(m. . The discrimination mask can be of any shape. C − 1) where ∗ denotes the convolution operator. n) (or. . . . C − 1) . n − C + 1) xj (m − 1. That is. T EXTURE V ECTOR C ONSTRUCTION It is well-known that texture is not only characterized by the gray level at a given pixel but also the gray-level pattern in a neighborhood region surrounding the pixel. xj (m. . All of the above-mentioned merits ensure the proposed method yielding high performance for both two-class and multi-class texture classiﬁcation problems. n) = xj (m. while other mask conﬁgurations can be easily extended. n). . 2004 ω= . n) = ω(m. and y j (m. . xj (m − 1. For simplicity. n) . . . n) where m and n are the spatial indexes of the texture pattern. n − q) (1) output feature for texture pattern xj (m. n) R−1C−1 p=0 q=0 = ω(p. 0) . Based on this intuition. n − C + 1) . xj (m. n) is of size R × C . n). n) and the local texture pattern of size R × C at position Thus. Using (m. n) = ω T xj (m. we also adopt the rectangular mask in this paper. n − C + 1) . . ﬁlter mask. II. 0) .4 ’kernel trick’ introduced in the proposed method ensures that the optimization does not depend on those undesirable assumptions of the input data required by LFD. what we want to compute is a discrimination mask ω(m. q) xj (m − p. Accordingly. and convolve it with xj (m. the feature extracted by the discrimination mask ω at position (m. n). . n) can be re-written as follows: yj (m. given several classes of texture patterns. we denote texture pattern j by x j (m. ω(1. (2) (3) DRAFT . . n) is the the lexicographically ordering of ω(m. yj (m. n) ∗ xj (m. the vector formulations can be obtained: ω(0. as commonly called in the literature). ω(R − 1.

Linear Fisher Discriminant Linear Fisher discriminant (LFD) is a well-known two-class discriminative technique. which ensures that the output (texture features) of each texture class could result in the maximum separation between different textures such that promising texture classiﬁcation results can be achieved. Gabor ﬁlter.. Most of the texture feature extraction methods such as AR model. III. ω here can be alternatively considered as the discrimination mask) which maximizes the following Rayleigh coefﬁcients [15] ω T SB ω ω T SW ω J(ω) = (4) where SB = (m1 − m2 )(m1 − m2 )T 2 lj SW = j=1i=1 (xi − mj )(xi − mj )T (5) September 7. we will show how to derive an optimized discrimination mask by exploring the principle of KFD.. x22 be the sets of training samples generalized from two 1 1 l l different texture classes. In the following. Each sample here is a vector which is formulated from texture pattern according to (2).5 Therefore. . Thus LFD is capable of performing feature dimensionality reduction for classiﬁcation.. In the following sections. LFD is given by the vector ω (based on the formulation stated in Section II. 2004 DRAFT .. x11 and χ2 = x2 . l1 and l2 are the number of training samples corresponding to each class. n) can be computed from the dot product between the mask coefﬁcients and the corresponding pixel values. It aims to ﬁnd the optimal projection direction such that the distance between the two mean values of the projected classes is maximized while each class variance is minimized. . (3) tells us that a feature value for a local texture pattern at position (m. nonlinear Fisher discriminant).. K ERNEL F ISHER D ISCRIMINANT A. Laws ﬁlter [2].e. let us begin with two-class texture classiﬁcation. which will be further extended into a kernel version (i. Let χ1 = x1 . The key issue of texture discrimination is to ﬁnd an optimal discrimination mask.. with some modiﬁcations can be ﬁt into the general vector formulation [14]. Let l be the total number of training samples of all classes. wavelet transform.. The following overviews the principle of LFD. because only one-dimensional features are extracted for two-class problem.

LFD can be applied in F (corresponding J(ω) = (8) φ φ where ω ∈ F . Consequently. the optimal discriminative texture features can be directly computed using ω ∗ by the projection deﬁned in (3). B. However. SB and SW are the corresponding between-class and within-class scatter matrices. Consider there is a feature mapping φ which maps the input data into a higher-dimensional to nonlinear operation in the input space χ).6 are between-class and within-class scatter matrices. In this case. i. 2004 DRAFT . Accordingly. It is equivalent to maximizing the following criterion: φ ω T SB ω φ ω T SW ω inner-product space F . its optimality heavily depends on the assumption that all the classes have equal covariance matrices. that is. ω becomes a nonlinear texture discrimination mask. i i=1 (6) The optimal discrimination mask can be computed explicitly in a closed form by the following: ω ∗ = arg maxJ ω = S−1 (m1 − m2 ) W (7) ω ∗ is an optimal linear feature extractor. It is obvious that the real-world data are usually not linearly separable and do not meet such a requirement. formed in F . respectively. φ SB = (mφ − mφ )(mφ − mφ )T 1 2 1 2 φ SW = 2 lj j=1 i=1 (φ(xi ) − mφ )(φ(xi ) − mφ )T j j (9) with mφ = j 1 lj lj j i=1 φ(xi ). φ : χ → F . To overcome this limitation. Its implementation can be achieved by employing the ’kernel trick’ introduced by Vapnik [16]. respectively. September 7.e. kernel Fisher discriminant (KFD) has been introduced to ﬁnd a nonlinear projection direction for two-class problem [13]. Kernel Fisher Discriminant LFD has a very good connection to the optimal linear Bayesian classiﬁer in the sense that the optimal projection direction corresponds to the optimal Bayesian classiﬁer [12].. m j is deﬁned by mj = 1 lj lj xj .

Thus. 2004 DRAFT . y in F . the denominator of (8) can be reformulated as follows: ω T Sφ ω = αT Nα W (14) where N = k(xn . the optimization αT Mα αT Nα T j=1. xj ) = αT µj k i=1 k=1 (12) where µi = j 1 lj lj j k=1 k(xi . the numerator B of (8) can be computed as follows: ω T Sφ ω = αT Mα B (13) where M = (µ1 − µ2 )(µ1 − µ2 )T . φ(xj ) k i=1 k=1 (11) where follows: φ(xi ). φ(xt ) = i=1 αi k(xi . the projection of a test pattern xt onto ω can be computed by l ω. and Kj is an l × lj kernel matrix of class j with (Kj )nm = of (8) is equivalent to ﬁnding the optimal value of α by maximizing the following criterion: J(α) = (15) The optimal vector α∗ can be computed by ﬁnding the leading eigenvector of N−1 M. we can compute the projection between the two j vectors in F as follows: ω T mφ = j 1 lj l lj αi φ(xi ). By introducing k a kernel function k(x. y) to represent the inner product x. xk ).2 Kj (I − Lj )Kj . (11) can be rewritten as ω T mφ j 1 = lj l lj αi k(xi . φ(xj ) k represents the inner product between φ(xi ) and φ(xj ). the solution of ω ∈ F must lie in the span of αi φ(xi ). Similarly. xt ) (16) September 7. ω can be formed by a linear combination of the mapped training samples in F as follows: ω= i=1 l From the theory of reproducing kernels [17].7 all the training samples in F . Thus by using the deﬁnition of Sφ in (9) and (12). (10) By combining (10) and the deﬁnition of mφ . xj ). Once α∗ is obtained. m I is the identity matrix and Lj is the matrix with all entries lj −1 . Thus.

The optimal discrimination mask can not be computed explicitly. This tells us that we only need to deﬁne a kernel form of an inner product instead of computing the explicit form of this mapping. n. As the dimensionality of F is usually much higher than the number of the training samples which could cause the matrix N being non-positive deﬁnite. 2004 (20) DRAFT . y) = (x · y)n − x−y σ 2 where σ . Fortunately. Since our objective is to derive the features by projecting the texture patterns onto the discrimination mask deﬁned by (3). y) = tanh(x · y + Θ) • Polynomial kernel: k(x. Commonly-used kernel functions are summarized as follows: • Gaussian kernel (radial basis function (RBF)): k(x. ﬁnding the optimal value of α is an ill-posed problem. That is. and Θ are the parameters of the three kernels. TWO . we use the former as in [13].CLASS TEXTURE CLASSIFICATION KFD is originally designed for the problem of binary classiﬁcation. In this paper. KFD can be constructed by selecting the proper kernel.8 Note that rather than computing the left-hand side of (16). we do not have to compute the explicit form of ω but the projection directly. Nr = N + rI (17) IV. given a texture pattern. y) = exp • Tangent hyperbolic kernel: k(x. the closed-form solution of maximizing (15) can be computed as follows: α∗ = arg max J(α) = N−1 (µ1 − µ2 ) α l (18) (19) ω ∗ = i=1 φ(xi ). the optimal discrimination mask is derived by ﬁnding the optimal value α ∗ . For the two-class texture classiﬁcation. its one-dimensional feature can be directly extracted by the following equation without using the explicit form of the mapping function φ: y = N−1 (µ1 − µ2 ) · Kt September 7. respectively. because we do not know the explicit form of mapping φ. Fortunately. the right-hand side can be much more easily obtained via a linear combination of the inner products which is independent of the mapping operator φ. consequently. which simply adds a multiple of the identity matrix or the kernel matrix K to N to guarantee that N is positive deﬁnite. Without considering the mapping φ explicitly. The commonly-used approach to solve this problem is to employ regularization.

in this section. For simplicity. Thus the decision function is simply deﬁned as follows: f (t) = arg min |yt − y j |. Suppose that we have c texture classes and each class has l j training samples. we use the simplest nearest-mean classiﬁer for classiﬁcation [18]. many real applications involve more than two kinds of texture. Therefore. . any column of the solution W must lie in the span of all training samples. where j = 1. the mean of each class y j can be computed.CLASS TEXTURE CLASSIFICATION In the previous sections.. and ω is a column of W. Using ω to represent any one column of W. Since the KFD is optimized via the maximum separation.. In F . and mφ = j 1 lj lj i=1 φ(xi ). However. Referring to (22). in this paper. Using yj (j = 1. Given a test texture pattern t. (24) September 7. M ULTI .9 where Ki = k(xi . In this case. 2) to represent the outputs of KFD for the training samples of texture class j . we have discussed the problem of two-class texture classiﬁcation based on KFD. we rewrite the Fisher criterion as follows: J(W) = φ SB = φ SW c c j=1 lj φ | W T SW W | φ | W T SB W | (22) lj (mφ − mφ )(mφ − mφ )T j j (φ(xi ) − mφ )(φ(xi ) j − m φ )T j (23) = j=1i=1 with mφ = 1 l l i=1 φ(xi ). Multi-class KFD is a natural generalization of the kernel Fisher discriminant based on multiple discriminants [15]. we deﬁne W to represent such a matrix. j (21) V. we have: ω= i=1 l αi φ(xi ). c. which accounts t for the minimum classiﬁcation (Bayesian) error according to the probability theory. ω becomes a projection matrix which contains c − 1 directions. 2004 DRAFT . its corresponding KFD feature is yt . thus any classiﬁer can be the candidate. xt ). hence. it has fully utilized the discriminative information expressed by the training samples.. More than one ﬁlters are usually needed in the multi-class texture classiﬁcation. the selection of the classiﬁers for classifying the obtained texture features is not an issue. we generalize the two-class KFD algorithm to deal with the multi-class texture classiﬁcation. Therefore. To be more clear.

Usually. The complexity of the computation of multi-class texture classiﬁcation is thus reduced almost as the same as that of two-class texture classiﬁcation. The optimal matrix A∗ can be computed by ﬁnding the (c − 1) leading eigenvectors of N−1 M corresponding to the none-zero eigenvalues. αc−1 ]. In this paper. This projection can be computed as follows: y = A ∗ T Kt (28) texture classiﬁcation problem.. D(·) is the distance function of the feature vector of yt and yj . A∗ = arg max A (27) where A = [α1 . For example. the decision function for a given texture pattern t thus becomes the following form: f (t) = arg min D(yt − yj ) t i where Kt = k(xi . Once A∗ is obtained. xk ). by using typical one-against-the-rest technique with two-class KFD. since we have to ﬁnd c ﬁlters by two-class optimization. within the framework of the multi-class KFD. the computation cost will be c times of that of the two-class texture classiﬁcation problem. more different textures involve higher computational complexity.. each of which discriminating one texture against the rest. the goal of multi-class KFD is to ﬁnd | AT MA | | AT NA | KKT − µi = 1 l l k=1 k(xi . we simplify it as the Euclidean distance. September 7. The computation of M and N requires only kernel computations. However. for a given texture pattern xt . we can project each of the class mean onto an axis of F by using only the dot product: ω T l lj mφ j 1 = lj αi k(xi . and y is a feature vector of size (c − 1) × 1. 2004 DRAFT . as the two-class (29) where yj is the mean feature vector of class j .10 Similarly as the two-class kernel Fisher discriminant. we can map it to a (c − 1) dimensional discrimination space spanned by the columns of A ∗ . xj ) = αT µj . Thus we need c times two-class optimizations when using typical one-against-the-rest technique. which are equivalent to the optimization of the wellknown Rayleigh coefﬁcients.. N = lj j k=1 k(xi . the optimization is only one shot and furthermore the solution is in a closed form. xt ). . k i=1 k=1 (25) It follows that φ ω T SB ω = αT Mα φ ω T SW ω = αT Nα (26) c T j=1 lj µj µj . xk ). where M = and µi = j 1 lj c T j=1 lj (µj − µ)(µj − µ) . for a c-class texture classiﬁcation problem. Thus. Similarly.

+1] to ensure that the textures are not trivially discriminable simply based on the local mean or local variance. the efﬁcacy and effectiveness of KFD-based texture feature extraction approach for texture segmentation are evaluated by performing supervised segmentation based on several test images with various texture complexities. The training data are obtained by randomly September 7. from 5 × 5 to 21 × 21 (all in odd numbers). Therefore. resulted from mis-classiﬁcations. E XPERIMENTAL R ESULTS In our experiments. The improvement of morphological ﬁltering on the ﬁnal results depends on the size and the shape of uniform texture of 5% is achieved in terms of the error rate with respect to the ground truth segment. the performance of our method for texture classiﬁcation are tested on different textures with different window sizes. Each image is globally histogram equalized and normalized to [-1. Different portions of the input patterns of each texture class are randomly selected and used for training KFD. The size of these test images is 256 × 512. The size of the texture-pattern window (R × C in (1)) will affect the classiﬁcation performance.. In this paper. An average improvement from two-class texture images to as many as nine-class texture images as shown in Fig. Each texture image has a size of 512 × 512. The texture sources used in our experiments range and MIT Vis-Tex database [20]. 2. We avoid using the texture patterns on the texture borders for training because these patterns are not suitable for training. Two-class Texture Image Several texture pairs as shown in Fig.11 VI. A. 256 grey levels). 1 are tested for binary classiﬁcation using the proposed method. respectively. 1 and Fig. The reported error rates are computed based on the decision function in (29) without applying any post-processing. We experimentally set the size of the ﬁltering window as 5 × 5.. we focus on the measurement of classiﬁcation error as a yardstick of performance evaluation. 2004 DRAFT . The test images are constructed based on the textures from two commonly-used natural texture image databases: Brodatz album [19] pixel (i.g. morphological ﬁltering [11]) over the map is able to effectively remove them in order to get a higher classiﬁcation accuracy. VII. Applying a post-processing technique (e. P OST-P ROCESSING Classiﬁcation map often contains speckle noise. One concern of using this technique is how to choose a proper window size for it will introduce a tradeoff between minimizing the total error and identifying the border between two textures correctly.e. with 8 bits per regions.

Leaves.12 (a) (b) (c) Fig. D24.0005.0016. D9. For polynomial kernel. (a) (b) (c) Fig. (a) D4. Misc. and Sand. A set of experiments are conducted on the texture images in Fig. and D38 from [19].0019. Food. selecting 400 patterns from each texture class. 2004 DRAFT .0000 from [20]. the KFD texture feature extraction is performed based on 800 texture patterns for each texture pair.0005. and D57 are from [19].0007 and Food. 2. (d) Two-class texture images used in our experiments. D28. D19.0005 from [20]. D21.0009. The implementation of KFD needs to select a proper kernel function and tune the corresponding kernel parameters. (c) Fabric. Flowers. (c) D4. 1. Two important kernels are Gaussian kernel and polynomial kernel. (b) Fabric. D9. That is.0003. (a) D4 and D84 from [19].0000. Fabric. the choice of its degree n will dramatically affect the classiﬁcation performance. D37. Misc. D36. Multi-class texture images used in our experiments. which account for only 0. 1 with the window size of September 7. (d) D5 and D92 from [19].6% of the total available thousands of input texture patterns.0002. Fabric. (b) D28 and D29 from [19]. D19.

by using polynomial kernel. The observation coincides with some research in kernel theory. the results by Gaussian kernel are much more stable for both homogeneous and non-homogeneous textures in terms of error rates (3.7%. we can see that. another group of experiments are conducted. For an overall evaluation of the effect of both polynomial and Gaussian kernels in the proposed method. n = 2 is adopted in the follow-up experiments. respectively). 1(b) and 1(d) with 17 × 17 window. for non-homogeneous textures.1%. In these cases.3% and 22. of the input patterns). Moreover. From the table.5%. the error rates with (26. for higher degrees. This can be interpreted by the over-ﬁtting problem [12].9% using 19 × 19 window). This may lead to much degraded performance of polynomial kernel-based KFD on non-homogeneous textures. the classiﬁcation performances are degraded. This property of polynomial kernel may distort the optimal direction found by KFD. error rates are achieved by using the windowing sizes of 17 × 17 and 19 × 19. To show the discriminative properties of KFD-based texture features for discrimination by using Gaussian kernel. the distance of mapped training samples in F could become extremely large [12].13 17 × 17 and 19 × 19. and clearly shows that degree n = 2 achieves the best error rate for all test images as shown in Fig. the best these window sizes are suitable to capture the texture properties. 1(b) and 1(d). more accurate results are achieved by Gaussian kernel than by polynomial kernel overall. the parameter is selected as σ = 0. Our experiments show that these window sizes are effective in capturing texture properties. from Table II. It is worth to highlight that. which is also an optimal parameter setting for RBF-based support vector machine (SVM) [12][17]. 2004 . we also note that the training errors is zero1 . respectively) are signiﬁcant lower than that with polynomial kernel For Gaussian kernel. This suggests that We observe that.3 · N (where N is the dimensionality window. 1. 5.6%.2%. On the contrary. The same can also be observed from the results using 19 × 19 RBF kernel (5. It can be seen that. DRAFT September 7. This is overlooked in the literature [21]. for texture pairs shown in Fig. This can be demonstrated by the error rates shown in Table II. Table I tabulates the error rates produced by using polynomial kernels with different degrees. 7. Thus. especially for more scattered training samples in the input space resulted from non-homogeneous textures as shown in Figs. It has been shown that. for Figs. Table II summarizes the classiﬁcation error rates by these two kernels. the classiﬁcation accuracy of polynomial kernels is worse than that of RBF kernels.7% and 8. 1. The effect of applying different sizes of window is also evaluated in this experiment. we also plot the feature map of each texture 1 Note that zero training errors does not mean accurate classiﬁcation. and 8.

After applying post-processing. where each row corresponds to Fig.14 pair. Multi-class Texture Image In this section. 2004 DRAFT . for texture pairs: D5 and D92.3) (c.1).2) (a. However. are created for this experiment.4) (c. and (d.3) (b.1).1) (a.4) (b. The second column is the classiﬁcation results. our proposed method has achieved a rather low error rate of 8.6% using 17 × 17 window. as shown in Fig. 1(a).2) (d. the Gaussian kernel outperforms polynomial kernel and yields a successful discrimination between closely-resemble textures. it is very hard to discriminate even by Gabor features [2][21]. (b). Note that. B.1). The third column is the classiﬁcation results after post-processing. September 7.2) (b.1). Therefore. respectively.4) (d. 3. as shown in Fig. 2.1) Fig. the performance of multi-class KFD in the context of multi-class texture images is evaluated. The fourth column is the error maps in terms of misclassiﬁed pixels.2) (c.1) (c. From these feature maps. In conclusion. (c).1) (b. (b. The ﬁrst column corresponds to the feature maps of different texture pairs. respectively.4) Two-class segmentation results of texture images in Fig. and (d). 3 (a. it is a preferable choice of kernels for KFD-based texture classiﬁcation.3) (a. we can see distinct discontinuity occurs at the boundary between two textures within each texture pair. (a. Three multi-class texture images.3.3) (d. the proposed method achieves fairly promising segmentation results. (d. (c. respectively.

respectively.0 49.0 18. projection directions are needed to recognize all of these four textures in our proposed method.6 23.8 23.2 10.3 8. Only three September 7. 1 (a) Fig.1 49.6 50.5 7. 2.3 8.8 22.8 49.3 24.6 13.9 5.6 19.9 26.5 19.0 18. Figs.1 2.6 49.6 7.2 12.0 49.6 49.3 17 × 17 4.5 19 × 19 47.7 9. 2(b) and 2(c).3 E RROR RATES (%) OF TWO .5 24. To understand the discrimination behavior of each feature component extracted by multi-class For Fig.9 30.9 2.1 17.6 33.6 12.8 15.4 13 × 13 6.3 33.7 12.2 7. 1 (b) 17 × 17 46. the lowest error rate obtained is 6.7 22.3 5.2 7×7 16.2 5.CLASS TEXTURE IMAGES USING VARIOUS WINDOW SIZES Window Size Kernel RBF Image Fig.6 49.6 49.1 34.6 3.1 8. while a database of 3600 training samples for Figs.4 20.7 49. 1 (a) Degree. 2(a) shows a 512 × 512 test image composed of four Brodatz textures.7 28.5 43. 2004 DRAFT . Thus.6 49.3 24.2 Fig.7 26.3 19 × 19 50.6 49.9 34. For all of these test images.6 11 × 11 8.5 26.8 Fig.1 11.0 14.6 11.6 33.7 50. 1 (c) 17 × 17 47.15 TABLE I E RROR RATES (%) WITH 17 × 17 AND 19 × 19 WINDOWS USING DIFFERENT POLYNOMIAL DEGREES Fig.6 49.3 49.4 7.1 19 × 19 49.7 22.5 15. a database of 1600 training samples are set up for Fig.8 10. 1 (c) Fig.6 13.2 18.5 10.3 7. 400 texture patterns are randomly selected from each texture class for training.6 29.7% when using 19 × 19 window.7 5.8 9×9 11. 1 (b) Fig. 1 (d) 17 × 17 48.6 TABLE II Fig.1 49. 2(b) and 2(c) are two 384 × 384 test images composed of nine texture images from Brodatz album and MIT Vis database.2 19.6 Fig.3 11.1 30.0 49.6 10.6 49.4 25. 1 (b) Fig.6 3.7 16. 2(a).6 24 15 × 15 5.5 22.1 11.6 49.7 49.7 2.1 17.7 20. Table III tabulates the error rates for each of the test images.5 22.7 7.2 15.4 30.0 49.0 38. n 1 2 3 4 5 6 17 × 17 49. 1 (c) Fig. 1 (a) Poly (n = 2) Fig.4 6.0 23.6 49.5 19 × 19 3. 1 (d) Fig. respectively.7 19 × 19 47. 1 (d) 5×5 20.8 14.6 28.6 21 × 21 4.6 8.6 26.8 34.

02 0 −0. the four textures can be well separated by the extracted KFD features. (f) error map shown in terms of misclassiﬁed pixels. Hence.07 0.04 0 −0. 5.13 0. (a) (b) (c) (d) (e) (f) Fig. 2 (a). 4(a)-(c). respectively. 2004 DRAFT .1 0. while the combination of the three ﬁlters can give a successful discrimination among these four textures. 3-D scatter plot of KFD features for four-texture image in Fig. (a)-(c) feature maps resulted from three feature components.16 nonlinear texture ﬁlters (each column of A∗ ) individually.14 D009 D019 D004 D057 Fig.11 0. 2 (a).03 −0.09 0. 5. Although these four textures are visually similar as shown in Fig. A 3-D scatter plot is shown in Fig. Four-texture segmentation results for the test image Fig. a very distinct discrimination can be observed in the scatter plot. A very interesting observation is that each ﬁlter tries to discriminate one texture class from the others. (d) classiﬁcation results. which presents some insights of the KFD texture features.01 −0. 4. September 7. individually.02 −0. 0.12 0.02 −0. 4.07 0. the three feature maps produced by each ﬁlter are shown in Figs.04 −0.08 0.06 −0.05 −0. (e) classiﬁcation results after applying post-processing.

2 (b) Fig.1 13. So the proposed method also performs well in multi-class texture classiﬁcation.4 13.8 11. For RBF-SVM. Two types of commonly-used SVMs are investigated in the comparison: polynomial-based SVM and RBF-based SVM (denoted as poly-SVM and RBF-SVM. 13.1 20. respectively). 2(c) using 21 × 21 window.9 21 × 21 7. (b) (c) (d) Nine-Vis-texture segmentation results for the test image in Fig. 2(b): (a) ideal segmentation result.9 9×9 11.8 13 × 13 7. (c) classiﬁcation results after applying post-processing. 2 (c) 5×5 23. (d) error map in terms of the misclassiﬁed VIII.3∗N [12]. Yet.7 12.6 17 × 17 6. 2(b) using 11 × 11 (a) Fig. satisﬁed error rates by window.17 TABLE III E RROR RATES (%) OF MULTI .2 14.8 12.2 20.2 23.3 13.4% for Fig. C OMPARISON In this section.3 11. pixels.6 11 × 11 8. The increasing number of texture classes will increase the difﬁculty to discriminate these textures. in this case.1 19 × 19 6.2 17. a comparative study is further conducted with the SVM-based texture classiﬁcation method [21].1 7×7 15.8 15 × 15 6. 2 UNDER DIFFERENT WINDOW SIZES RBF KERNEL Window Size Image Fig. 2004 1 0. (b) classiﬁcation result. 2(b) and 2(c) involve as many as nine textures. the parameter of RBF width σ is set as σ = September 7.CLASS TEXTURE IMAGE AS SHOWN IN USING F IG .4 12.6 Figs. where N is DRAFT .9 15. using our proposed approach using RBF kernel are achieved as 11. Texture classiﬁcation with the two types of SVMs is performed with different settings of parameters. 2 (a) Fig.7 13.5 16.6% for Fig.5 30. 6.4 20.

and they are suitable to capture texture properties as indicated in our previous study.18 (a) Fig. [21]. The 17×17 and 19×19 windows are selected for this study. Classiﬁcation error rates with these methods are summarized in Table IV. 2004 DRAFT . 2 (c). Accordingly. Overall. Furthermore. because most of the lowest error rates are achieved with these windows. (b) classiﬁcation result. pixels. respectively. and elegant way for both two-class and multi-class texture classiﬁcation problems. a texture classiﬁcation method based on the extracted texture features is described. We can see that the improvement by using the ”kernel trick” is very distinct. a nonlinear discriminative texture feature extraction method based on kernel Fisher discriminant (KFD) is proposed. 7. Form this table. (c) classiﬁcation result after applying post-processing. Accordingly. (b) (c) (d) Nine-Brad-texture segmentation results for the test image in Fig. RBF-KFD gives the lowest error rates among all of these methods listed in Table IV. The derived optimized closed-form solution gives a simple. (a) ideal segmentation result. the proposed KFD-based method outperforms the other texture discrimination methods in our comparison. For poly-SVM. The texture vector conﬁgurations follow the method presented in Section II. powerful. compared to SVM-based texture classiﬁcation method with different September 7. we also incorporate the classiﬁcation results by using the LFD-based texture classiﬁcation method. IX. the regularization term C is set as 100 for both poly-SVM and RBF-SVM. The implementation of SVM (LIBSVM) is used in this comparative study [22]. This also indicates that the textures studied in this paper are not linearly discriminable. we can see that the performance of RBF-SVM is much better than that of poly-SVM (degree equals 5) in terms of classiﬁcation error rates. C ONCLUSION In this paper. In order to understand the merits of our proposed kernel-based texture discrimination method. (d) error map in terms of the misclassiﬁed the dimensionality of the input texture patterns. the degree of SVM is set as 5 for texture classiﬁcation which is suggested by Kim et al.

etc. no.19 TABLE IV C OMPARISON OF ERROR RATES (%) Fig.” in Handbook Pattern Recognition and Computer Vision.7 48.2 4. 235–276. LFD.2 6. 1(d) 17 × 17 8. Wang (Eds. Yet. Shepp.1 19 × 19 7. L. Frisch. A. At moment.6 Fig.BASED TEXTURE CLASSIFICATION METHODS Fig. vol. 291–310. 1973.6 19 × 19 3.5 49. 1999.” IEEE Trans.” Perception. this is not our focus of this paper.1 9. pp. further study will be extended to unsupervised texture classiﬁcation.8 50. no. Julesz. “Texture analysis. and segmentation of remote sensing images.5 49. Pau. N.6 10.0 Fig. Note that the proposed method is a supervised approach. C.7 47. Hence.6 49. [3] B. [2] T. L. promising classiﬁcation results with the smallest error rates are obtained by using the proposed method. R EFERENCES [1] M. SVM-. 21.7 5. our approach use rectangular region of support for each texture pixel.9 21. Similar observations and arguments can be found in recent impressive works in [21] and [23].9 9.2 22. 2004 DRAFT . Possible choices may include the use of k-means to automatically select the training samples for each texture class. 1(c) 17 × 17 9. I.1 19 × 19 8. September 7. 1(a) Method RBF-KFD RBF-SVM Poly-SVM (p = 5) LFD 17 × 17 4. Gilbert. pp.6 49. Rotation invariance can be achieved using circular region of support.8 19 × 19 5. Singapore: World Scientiﬁc.7 kernel functions. 4.7 46. Tuceryan and A. The automatic determination of the number of classes should also be considered in unsupervised classiﬁcation. and H.7 USING KFD-. E. “Filtering for texture classiﬁcation: A comparative study. We will investigate this in our future research. and P. Chen. 1(b) 17 × 17 5. “Inability of humans to discriminate between visual textures that agree in second-order statistics – revisited. document segmentation. vol. Jain. pp.6 9. The combination of such work and our approach is expected to achieve good performance in recognizing textures across a wide of separate views.). We also demonstrate that it is possible to extract good discriminative texture features without apply ﬁltering on the image as a preprocessing step. 1993.5 47.7 6. Pattern Analysis Machine Intelligence. L. Husoy. It is also interesting to note that there are some recent work aiming to achieve invariance using local afﬁne regions [24] [25].6 50.2 49. 2. Randen and J. 391–405. Some of the applications of the proposed method may include object classiﬁcation based on textures.7 50.7 49.2 47.

2001. Fan. pp. 1998. no. Systems. vol. [25] ——. “Visual pattern discrimination. IT-8. on Information Theory. and J. vol. Smola. 4. Lazebnik. “Unsupervised texture segmentation using Gabor ﬁlters. no. Conf. on Pattern Analysis and Machine Intelligence. and S. Boston. G.” Proc. no. Jung. Randen and J. [16] V. June 2003. vol. P. pp. Kim. B. 1975. 2002. MA: MIT Press. 8. vol.” IEEE Trans. 1998. Larsen. Textures: A Photographic Album for Artists and Designers. 34–43. 1167–1186. Cambridge. [10] A. no. and Cybernetics. Picard. no.” IEEE Trans. Sept. Shangmugam. 41–48.edu/vismod/. Academic press. Brodatz. pp. I. Julesz. Man. [22] C. Introduction to statistical pattern recognition. 1973.” IEEE Trans. Regularization. [9] O. “Support vector machines for texture classiﬁcation.” Pattern Recognition. 1978. C. on Pattern Analysis and Machine Intelligence. J. Chang and C. vol. Hu. G. Pattern Classiﬁcation. 319–324. 9. no. on Image Processing. vol. [11] A. 7. 2002. 18. [18] K. April 1999. Sch¨ lkopf. 192–202. and H. S. Douglas (Eds. Pattern Recognition. vol. http://www.. 610–621.-J. O. pp. “Fisher discriminant analysis with kernels. [20] MIT Vison and Modeling Group. Pattern Analysis Machine Intelligence. and K. Fukunaga. Zisserman. pp. 2. and randomness: Wold features for image modeling and retrieval.” Neural Computation. Sch¨ lkopf. Faugeras.” IEEE Trans. Kim. no. Husy. IEEE. “A sparse texture representation using afﬁne-invariant regions. July 1996. Hart. Karu. R¨ tsch. 13. M¨ ller. “Learning texture discrimination masks. B. pp. M¨ ller. 571–582. 12. Duda. [12] B. Optimization o and Beyond. 1542–1550.-R. WI.-C. 2001. pp. 1991. on Pattern Analysis and Machine Intelligence. Nov. [8] F. “Texture classiﬁcation: Are ﬁlter banks necessary?” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. pp. 1962. 2. Feb. and L. Z. [13] S.” IRE Trans. K. 549–552.-H. no. 24. II. Varma and A. 24. J. 691–698.” IEEE Trans. 11. Feb.” in a o u Neural Networks for Signal Processing. Haralick. [21] K.. 1999. Jain and F. “Periodicity. Vapnik. [23] M. 2. 232. “Constructing descriptive and discriminative a o u nonlinear features: Rayleigh coefﬁcients in kernel feature spaces. E. New York: John Wiley and Sons Inc. 2004 DRAFT Madison. H. [7] F. 1991.” Submitted to IEEE Trans. [19] P. Mika. and D. Schmid. Liu and R.” Scientiﬁc American. [14] T. Farrokhnia. Statistical Learning Theory. June 2003. [5] ——. Learning with Kernels: Support Vector Machines. “Texture analysis and classiﬁcation using a human visual model. R¨ tsch. [15] R. “Classifcation of rotated and scaled textured images using gaussian markov random ﬁeld models. 1966. J. 5. Mika. E. May 2003. September 7. “Textural features for image classiﬁcation. directionality. “Experiments in the visual perception of texture. 84-92. Y. J. Wilson. Inc. 18. Jain and K. J. 623–628.-R. Dinstein. vol. Weston. 25. 195–205. 1990.media. G. “Training u-support vector classiﬁers: Theory and algorithms. Kyoto. vol. 722 –733. on Pattern Analysis and Machine Intelligence. [24] S. and M. New York: Wiley-Interscience. pp. vol. Sch¨ lkopf and A. 2119–2147. no. pp. Patel. “Texture segmentation using ﬁlters with optimized energy separation. pp. “A sparse texture representation using local afﬁne regions. Pattern Analysis and Machine Intelligence. Cohen. vol. IEEE Int.). . Weston. [17] S. K. vol.” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.” IEEE Trans. Ponce. pp.20 [4] B. Stork. and K.mit. [6] R. 3. New York: Dover. 1996. pp. Park.” IEEE Trans. Lin.

21 2004. September 7. 2004 DRAFT .

- Face Detection
- An Appraisal of Off-line Signature Verification Techniques
- ENGSTRUCT-D-15-00999.pdf
- Forecasting Power Output of Photovoltaic.pptx
- 12 Deepak Fuzzy Logic SVM
- Review on Blood Vessel Detection in Retinal Image
- Face Detect
- Estimation of Scour Downstream of Spillways
- DocLayout ITC Irst TR9703 09
- Use What You Choose Presentation
- Front and Rear Vehicle Detection Using
- psat
- 10.1.1.174.977.pdf
- 2013 Ieee Paper
- smo
- Satish
- 034
- Segmentation of Connected Red Blood Cells Based on Distance Per Displacement Ratio Maximization Criterion
- e i Gen Movements
- Ni Block
- Adversarially Learned Inference
- ijsrp-p2480.pdf
- ppt file
- Raizada_Quantifying the Adequacy of Neural_Cereb Cortex_2010
- Research Paper
- CIKM2009
- Machine Learning 3
- Feature-based laser data simplification for low polygon visualization
- Fast Similiarity Search for Learned Metrics (2009)
- Svm Classifier Algorithm for Data Stream Mining Using Hive and R

- Lemon Disease Detection Using Image Processing
- UT Dallas Syllabus for mkt6337.501.11f taught by B Murthi (murthi)
- Contour Free Level Set Method
- A brief Analysis on Offline Character Recognition in Malayalam Scripts
- tmp9F03.tmp
- Hand Gesture Recognition using Virtual Canvas
- tmpDF60.tmp
- IJSRDV3I110455.pdf
- A Novel Management Framework for Policy Anomaly in Firewall
- Analysis And Detection of Infected Fruit Part Using Improved k-means Clustering and Segmentation Techniques
- Intensification of old documents and photos by Digital Image Processing Techniques
- A Mining Cluster Based Temporal Mobile Sequential Patterns in Location Based Service Environments using CTMSP Mining
- Background Foreground Based Underwater Image Segmentation
- tmp904.tmp
- tmp8224.tmp
- tmp2A7B.tmp
- Depth of Field Image Segmentation Using Saliency Map and Energy Mapping Techniques
- Tumor Detection using Particle Swarm Optimization to Initialize Fuzzy C-means
- Improved remote sensing image segmentation using fuzzy logic for dynamic statistical region merging algorithm
- Segmentation and characterization of skin tumors images used for aided diagnosis of melanoma
- A Comparative Study of Fuzzy Logic with Artificial Neural Network Techniques in Cancer Detection
- tmp2D02.tmp
- Survey on image segmentation methods from blood microscopic images for detecting leukemia
- A Survey on Text Segmentation Methods for Digital Images
- tmpC0CF.tmp
- Determination of unique Feature set for AM Bitewing Dental Radiographs as a Bio-metric information
- Trajectory Segmentation and Sampling of Moving Objects Based On Representativeness
- Analysis of Infected Fruit Part Using Improved K-Means Clustering Algorithm
- A Review on Image Segmentation using Clustering and Swarm Optimization Techniques
- tmp2A89.tmp

- Book1_mm
- TISS Paper...
- BA
- Debt Funds
- Grammar Supplement
- ES Submission Deadline
- financials
- Book1
- Time Table
- Group10 Eight Levers Strategy
- ES guideline
- Just for Laugh
- Book12
- At the Beginning of the Simulation
- Project Test
- Bookn
- TELIASONERA
- Data SBI Growth%
- Analysis
- Book1
- Bookn
- data_SBI
- Bugs in Prowess
- Book1 Industry Data
- Book1
- for tcs
- Book13
- Business Creditcard
- Kumudam Snehidi
- a

Sign up to vote on this title

UsefulNot usefulClose Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Close Dialog## This title now requires a credit

Use one of your book credits to continue reading from where you left off, or restart the preview.

Loading