You are on page 1of 6

Classification of Prehensile EMG Patterns With Simplified Fuzzy ARTMAP Networks

Marko Vuskovic and Sijiang Du

Department of Computer Science San Diego State University San Diego, CA 92182-7720

Abstract Simplified fuzzy ARTMAP networks (SFAM) typically generate a large number of output neurons and require large number of input neurons due to the input complementation. Here is proposed a modification of SFAM, which uses activation and matching functions based on Mahalanobis distance. This modification considerably reduces the network size and increases the efficiency in training and classification. The new network has shown an excellent performance in classification of prehensile motions based on EMG patterns.

I. INTRODUCTION The work presented in this paper is motivated by the need for stronger classifiers that would help to implement the human-machine interface for the new generation multifunctional hand prostheses. The classical EMGdriven1 prosthetic devices are based on a single EMG channel, which activate closure or opening of a single degree of freedom device. The new generation prostheses will use artificial hands with several degrees of freedom, thus enabling multiple prehensile functions. The control of such devices requires more elaborate human-machine interfaces, capable of recognizing a multitude of volitional prehensile movements, making the hand control transparent in a sense that the subject can move the hand in a most natural way as if he/she would do with a healthy hand. This would require usage of multi-site EMG electrodes and elaborate approaches for real-time pattern recognition. The pioneering work done by Graupe [8], Saridis [15] , and Hudgins [11] has shown that such approach is possible. Although their work was focused on upper limb prostheses, the concepts were generally applicable to hand prostheses [6], [10], [14], [17]. In an earlier effort at SDSU [18], an encouraging level of the classification success rate was achieved by using a simple statistical classifier with a simple feature extraction applied on a four-channel surface EMG electrode set. The length of EMG samples was 300400 milliseconds, which is rather too long for real-time hand control that permits at most 200 milliseconds for feature extraction and classification. For this purpose far more elaborate feature extraction and feature classification methods have to be used. Unfortunately the two-layer perceptrons with back propagation learning did not help very much [19]. It has proven to be very sensitive to the

amount and ordering of training patterns. The convergence of the training process was very long, while the success rate did not improve significantly. Besides, the back propagation networks are not very suitable for incremental learning, which is a crucial capability of such systems. Fortunately the relatively new class of networks, the fuzzy ARTMAP networks have offered a hope for the solution. Adaptive resonance theory (ART) networks originally proposed by Carpenter, Grossberg and other researchers at Boston University [1], [2], have offered new possibilities in pattern classification. While the originally proposed networks were designed for unsupervised learning and categorization of binary patterns, the newer proposal of fuzzy ARTMAP networks for supervised incremental learning and classification of analog patterns, made by the same group [3], [4], [5], has offered a powerful alternative to BP networks. The new paradigm provides a direction that leads to elimination of the deficiencies of BP networks listed above. We have originally used a simplified version of fuzzy ARTMAP proposed by Kasuba [12]. This network, like other fuzzy ARTMAP networks uses normalized and complemented inputs, which doubles the size of the classifier and makes it less desirable for real time applications. In addition, this network typically generates a relatively large number of output neurons. Therefore we have modified the activation and matching functions by using the Mahalanobis distance instead of classical norm of fuzzy AND function. This has significantly improved the classification success rate. The new algorithms are more computationally intensive, however they reduce the number of output neurons, which results in overall smaller computational times. This paper summarizes the experiments with new simplified fuzzy ARTMAP network applied to feature extracted in the previous work [18]. . II. CLASSIFICATION OF PREHENSILE EMG PATTERNS A. Prehensile Motions As opposed to the most of the research in the area of EMG-controlled multifingered prosthetic hands [6], [10], [14], [17], we focused on the synergetic control of the groups of finger joints that correspond to the basic prehensile motion types, rather than on the controlling of individual fingers or individual finger joints. As the basic

Electrically driven, based on Electromyographic signals

prehensile motion types we have adopted Schlesinger's classification of human grasps [16]: cylindrical grasp, spherical grasp, precision grasp (pinch), lateral grasp (key grasp), and hook grasp. The grasping motions performed during the recording of the EMG patterns were consistent with this classification (with the exclusion of the hook grasp). This has resulted in four major grasp types. In addition, we have also experimented with two sizes of cylindrical and spherical grasps, which totaled to six grasp types. B. Measurement of EMG Patterns The EMG patterns were recorded with the Muscle Tester ME3000, by Mega Electronics, Ltd. The experiments [18] were performed on several human subjects with healthy hand. Four electrodes were placed above the relevant extensor muscles on the upper forearm, which are associated with the specified grasp types: Extensor Pollicis Group, Extensor Communis Digitorum, (two electrodes), and Extensor Carpi Ulnaris. In order to make the motions natural, we have prepared objects like cylinders, balls, keys and pearls, which were then grasped by the subjects in repetitive grasp movements. An example of an EMG pattern, obtained at the second channel while grasping a large ball, is shown in Figure 1. It can be seen from the diagram, that the grasp has started approximately at 380 ms, while the preshaping phase has completed at about 600 ms. This means that, the preshaping phase, which is relevant for the hand control, takes about 200 ms.

a four-dimensional feature vector f = ( f 1 , f 2 , f 3 , f 4 ) .

Each element f i of the feature vector corresponds to an EMG channel.

Figure 2: Raw pattern after squaring

Figure 3: Squared pattern after passing through a moving average filter with Hamming window

D. Feature


Features that correspond to different grasp types group into different feature clusters. In order to perform successful pattern recognition the feature clusters should have small or no overlapping. Figure 4 shows a projection of the four-dimensional feature space obtained for a single subject, onto a two-dimensional plane. In order to get the best two-dimensional view of the clusters, we have applied Fisher-Rao transformation of the original feature space [7]. The figure shows four clusters that correspond to four basic grasp types defined above.

Figure 1: Raw EMG Pattern (Extensor Communis Digitorum)

C. Feature Extraction Using the EMG values directly as classifying features, would result into 4x200 inputs for the classifier. Therefore the raw EMG patterns have to be preprocessed and reduced to a much smaller number of features in order to reduce the dimension of the classifier and to filter out the noise and artifacts. There are many approaches for feature extraction that are used in such situations, however we have applied one of the simplest: the raw EMG patterns were squared (see Figure 2), then passed though a moving average FIR filter (Finite Impulse Response filter) with Hamming windowing function of size 300 ms. Resulting temporal signal is shown in Figure 3. The first oscillation of the averaged signal corresponds to the preshaping EMG burst, and the maximum of the first amplitude is used as a feature. Thus, the amount of approximately 800 samples was reduced to only four features, represented by

Figure 4: Feature clusters for four grasp types projected onto x-y plane

In order to view the amount of the overlapping, the clusters can be encircled into confidence ellipsoids, as shown in Figure 5. The confidence level used in Figure 5 was 2.15, which conforms to 95% probability of a feature to be within the corresponding ellipsoid. It should be noted that the overlapping shown in the figure is not final,

because there are other two dimensions, which cannot be seen here. Figure 5 also illustrates different "geometry" of clusters associated with different grasp types. For example, their elliptic approximation shows that the cluster ellipsoids have different shapes (eccentricity) and different orientation of their principal axes. This is an important fact, which will be discussed later in the paper.

which requires computation of the covariance matrix for each cluster: T 1 n Sj = x j (k ) c j )(x j (k ) c j ) . (4) ( n 1 k =1 This approach has given excellent results [18]. Statistical approach based on the minimization of the Mahalanobis distance works extremely well in cases when the clusters are elliptically shaped. However, for more general cluster shapes, stronger methods have to be used. An attractive possibility seemed to be the back propagation perceptrons, used in most of the EMG pattern recognition of prehensile motions [6], [10], [14], [17], and of upper arm motion [9], [11], [13]. We have also had a solid experience with various kinds of BP networks and various approaches in feature extraction for neural networks [19]. This experience can be summarized as follows: (a) The success rate for BP networks was in the best case 1-5% above the success rate of the Mahalanobis classifier; (b) BP networks are very hard to train, i.e. the convergence of BP training is very much affected by the size and ordering of the training set. In addition, the length of the training time is unpredictable, and the convergence is even not guaranteed; (c) BP networks must be structured before they can be used, i.e. the number of hidden layers and hidden neurons in each hidden layer must be determined in advance; (d) There is no organized method available to structure a BP network, it is a matter of trial and error, which again depends on the size and ordering of the training set; (f) The BP networks are not adaptive, and are quite inefficient in incremental learning.
III. SIMPLIFIED FUZZY ARTMAP NETWORKS (SFAM) In this research, we have used a modification of the Simplified fuzzy ARTMAP network proposed by Kasuba [12]. The algorithm proved to be very efficient and fairly robust in terms of sensitivity to the size and ordering of the test set. The algorithm traditionally uses normalized and complemented inputs:

Figure 5: Projected feature clusters encircled by 95% confidence ellipsoids

In the following discussion we will use term "classifier input", or "input pattern", x = (x 1,x 2 ,..., x d ) , instead of feature vector f, since the features usually undergo further linear and/or nonlinear transformations before they are presented to the input of a classifier. A sample of an input pattern will be denoted as x ( k ) , while the sample set which belongs to the grasp type j will be denoted as: X j = {x j (1), x j (2),..., x j n } .
( )

Set X j will also be called "cluster j". The goal of a classifier is to determine the association of an unknown pattern y with one of the predefined clusters. Essentially, this association is determined by some optimization process. For example, the simplest classifier is a statistical classifier that minimizes the Euclidian distance:
min R(y, X j ) = min (y c j ) (y c j ),
T j j


where c j is the center of the cluster j:

1 n c j = x j (k ) . n k =1

1, f 2 ,...f d ,1 f 1,1 f 2 ,...,1 f d ) , f = f f . x = (f


The activation and matching functions were defined as follows:

t j = x w j / ( + w j ), mj = x wj / x ,


However, just a look at Figure 5 would discourage us from using the Euclidian distance for classification of EMG patterns. Euclidian distance would be justified if the clusters were spherically shaped. Therefore, more appropriate approach would be to use the Mahalanobis distance:
min M(y, X j ) = min (y c j ) S j 1 (y c j ),
T j j

where w j are current values of templates (weight vectors) associated with output nodes j. The updates of templates that belong to resonant domain are represented here as an assignment statement: w j := (1 ) w j + x w j . (6) where is the learning rate. The operator x w j =


min (x i , w j i ) used in (5) and (6) defines "fuzzy AND-

ing", which assumes positive, normalized values of the inputs. The results obtained with SFAM are presented and discussed in section V.

As mentioned above, the traditional SFAM requires normalization and complementation of input patterns. This doubles the dimension of the network and introduces a considerable computational overhead. Therefore we have explored other options which do not require normalization and complementation of inputs.
A. SFAM Based on Euclidian Distance

recurrent versions of (2) and (4). The recurrences add to the computational overhead. So does the need to invert the covariance matrix at each update. However, as will be shown in section V, the improved coverage of output neurons significantly reduces the number of output neurons and therefore the need for these computations. This has resulted in a further reduction of time for training and for classification. The success rate has also improved dramatically by another 8 to 17 %.

At first, we have used the Euclidian (squared) distance for activation and matching functions: x = f,
t j = R(x , X j ) = (x w j ) (x w j ),


m j = t j / max(x x, w j w j ),

Consequently, the resonance domain has to be determined by the minimization of the activation function, rather than by its maximization. The update function was: w j := (1 ) w j + x . (8) The templates w j have apparent geometrical interpretation as centers of clusters. The update function is essentially a recurrent expression for computation of the mean value of the cluster samples (2). As discussed later in section V, this approach has resulted in a considerable reduction of the processing time required for both, training and for classification. The success rate wasn't decreased; it has even been improved by one percent. The number of output neurons has however increased considerably, which can be explained by the "elliptic" nature of the clusters, shown in section II.D.
B. SFAM Based on Mahalanobis Distance

The results shown here were based on EMG signals recorded for a subject who has performed thirty grasps for each of the six grasp types listed in section II.A. Three SFAM were tested for two cases: six grasp types and four grasp types. In the latter case, grasping of large and small spherical objects was considered as a single category of spherical grasps. Similarly, grasping large and small cylindrical objects was considered as a single category of cylindrical grasps. The total amount of grasps for each grasp type was randomly divided into two groups, the training and the test set. The update functions (6), (8) and (10) were used in training session, while they were frozen in test session. All patterns from the training set were presented in a random order in each epoch. For each case there were 100 epochs. In each epoch, there was a different random separation of data into training and test sets. The ordering of training patterns was also random in each epoch. The classification success rate was computed for each epoch separately, then averaged across all epochs. The results for the two cases were presented in tables below. As shown, the new algorithm based on Mahalanobis distance has achieved much higher success rate than the classical SFAM. In addition, the new algorithm has proven to be three to almost six times faster in training and in testing as well. The number of output nodes was significantly smaller than in classical SFAM case.
TABLE I COMPARISON OF SFAM NETWORKS, SIX-CATEGORY CASE Classical SFAM Average classification hit rate Avr. number of output nodes Avr. learning time2 (per pattern) Avr. classification time (per pattern) 61.1 % 24.3 61.9 ms 61.0 ms Euclidian Activation Function 60.1 % 53.7 22.2 ms 21.7 ms New Algorithm 77.6 % 7.0 10.6 ms 5.9 ms

A logical course of action was to extend the "coverage" of output neurons, by applying the same idea as in statistical classification discussed in section II.D. Therefore the equations for the activation and matching function are given as follows:
t j = M(x , X j ) = (x w j ) S j 1 (x w j ),

m j = t j / max(x x, w j w j ). This time there are two quantities to be updated for each output node in the resonant domain, the templates w j (the


centers of clusters), and the covariance matrices S j that characterize the elliptic approximation of the clusters: w j := (1 ) w j + x , T (10) S j := (1 ) S j + (1 2 ) (x w j )(x w j ) . Equations (10) are derived in Appendix. As it will be shown, the two assignment statements represent the

All time measurements were performed on an unoptimized Matlab prototyping code executed on a Pentium II processor. With a proper code optimization, using C/C++ and executing on a Pentium 4 machine the performance is expected to be more than 50 times better.

TABLE II FOUR CATEGORY CASE Classical SFAM Average classification hit rate Avr. number of output nodes Avr. learning time (per pattern) Avr. classification time (per pattern) 85.7 % 9.4 27.9 ms 24.7 ms Euclidian Activation Function 86.53 % 30.1 13.3 ms 12.9 ms New Algorithm 94.6 % 5.2 9.1 ms 4.8 ms

and incremental approximation of a piecewise continuous function) in order to verify the generalizability of the results.

Dr. Antonio Ulloa from Cognitive and Neural Systems Department, Boston University, had a major impact on our decision to experiment with fuzzy ARTMAP networks, which we would like to acknowledge gratefully.

The results shown above are clearly superior to ones achieved with BP networks in [19] in terms of the success rate and specially the speed of the training process. Unfortunately we were unable to provide an adequate quantification, since in [19]are used larger sequences of EMG signals (over 400 ms), while in this paper the sequences were less than 300 ms, which made the classification much harder.

Recurrent computation of covariant matrix

Consider a cluster with n samples (the cluster subscript is omitted for simplicity): X n = {x (1), x (2),..., x n }
( ) ( )

A new version of simplified fuzzy ARTMAP network has been introduced, which is based on Mahalanobis distance used in activation and matching functions. The Mahalanobis distance requires an additional computation of inverse covariance matrix for each resonant output node. The recurrent computation of the covariance matrices and their inversion has introduced some computational overhead, however the algorithm has resulted in a significant reduction of output nodes, which made the algorithm overall faster in training and in classification, compared to the traditional SFAM. The new algorithm also yields a significantly higher classification success rate. The experiments were conducted with real EMG patterns recorded from a human subject in an earlier work. This experiment has definitely proven to be more stable and more efficient than the back propagation perceptrons. The further research will concentrate on two aspects of EMG pattern recognition: 1) more elaborate methods for feature extraction, and 2) investigation of the incremental learning characteristics of the newly proposed network. In this work each EMG electrode site has generated a single feature per grasping motion. We believe that feature extraction methods based on Short Time Fourier Transform with dozens of features per EMG channel will enable better discrimination of feature clusters, and consequently higher classification success rates [6]. The performance of the new SFAM network justifies the usage of the increased number of features, which would be difficult to use with classical SFAM, and practically impossible with the back propagation networks. In addition to the application of the new approach to EMG patterns, we will also test the approach with the standard benchmarks (Circle-in-the-square, two spirals,

The corresponding cluster center and covariance matrix are given as: 1 n c n = x (k ), (11) n k =1 T 1 n Sn = x (k ) c n )(x (k ) c n ) (12) ( n 1 k =1
( ) ( ) ( ) ( )

Suppose now an addition of a new sample to the cluster:

X (n +1) = {x (1), x (2),..., x n , x (n +1) }
( )

In order to give a higher weight to the new sample, we can add m identical copies of the new sample:
X (n +m ) = {x (1), x (2),..., x n , x (n +1), x (n +1),..., x (n +1) } (13)
( )

Consequently, the new cluster center will be defined as:

c (n +m ) = 1 n +m (k ) n 1 n (k ) m x = x + x (n +1), n + m k =1 n + m n k =1 n +m

or after substituting (11): n m c (n +m ) = cn + x (n +1) . n +m n +m If we express the learning rate in terms of n and m: m n = , 1 = , n +m n +m
( )





becomes: c (n +m ) = (1 )c n + x (n +1)
( )


Equation (16) can also be written as an assignment statement: c := (1 )c + x (17)

Similarly, the updated covariance matrix for the extended cluster can be written us: # n +m 1 S (n +m ) = x (k ) c (n +m ) ) , ( n + m 1 k =1



G.A. Carpenter, S. Grossberg, N. Markuson, J.H. Reynolds and D.B. Rosen, Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps, IEEE Trans. on Neural Networks, 3(5):698-713, 1992. K. A. Farry, I. D. Walker and R. G. Baraniuk, Myolectric Teleoperation of a Complex Robotic Hand." IEEE Trans. On Robotics and Automation, Vol 12, No. 5, October 1996, pp. 775788. R. Gnanadeskian, "Methods for Statistical Data Analysis of Multivariate Observations", John Wiley & Sons, New York, 1977, pp. 82-120. Graupe and W. K. Cline, Functional Separation of EMG Signals via ARMA Identification Methods for Prosthesis Control Purposes, IEEE Trans. System, Man Cybernetics, Vol. SMC-5, pp. 252-259, March 1975. Hiraiwa, K. Shimohara and Y. Tokunaga, "EMG Pattern Analysis and Classification by Neural Network," 1989 IEEE Internat. Cong. On Systems, Man, and Cybernetics, Cambridge, Nov. 1989, pp. 1113-1115. Han-Pan Huang and Chun-Yen Chen: "Development od a Myoelectric Discrimination System for Multi-Degree Prosthetic Hand," Proc. of the 1999 International Conference on Robotics and Automation, Detroit, May 1999, pp. 2392-2397. Hudgins, P. Parker and R.N. Scott, "A Neural Network Classifier for Multifunctional Myoelectric Control, Annual Int. Conf. Of the EMBS, Vol. 13, No. 3, pp. 1454-1455, 1991. T. Kasuba, "Simplified Fuzzy ARTMAP," AI Expert, pp. 18-25, November 1993. M.F. Kelly, P.A. Parker and R.N. Scott, "The Application of neural Networks to Myoelectric Signal Analysis: A Preliminary Study," IEEE Transection on Biomed. Eng., Vol. BME-37, No. 3, pp.221-230, March 1990. D. Nishikawa, W Yu, H. Yokoi and Y. Kakazu, " EMG Prosthetic Hand Controller using Real-Time Learning Method." Proc. of the IEEE Conf. on SMC, Vol. 1 1999, pp. I 153-158. N. Saridis nd T. P. Gootee, "EMG Pattern Analysis and Classification for a Prosthetic Arm," IEEE Trans. on Biomedical Engineering, Vol. 29, No. 6, June 1982, pp. 403-412. Schlesinger, "Der Mechanische Aufbau der kunstlishen Glieder", in Ersatzglieder und Arbeitshilfen, Springer, Berlin, 1919. N.U. Uchida, A. Hiraiwa, N. Sonehara and K. Shimohara, "EMG Pattern Recognition by Neural Networks for Multi Fingers Control," Proc. of the Annual Int. Conf. of the Engineering in Medicine and Biology Society, Vol 14, Paris, pp.1016-1018, October 1992. M. I. Vuskovic, A.L. Pozos and R. Pozos, "Classification of Grasp Modes Based on Electromyographic Patterns of Preshaping Motions," Proc. of the Internat. Conference on Systems, Man and Cybernetics, Vancouver, B.C., Canada, October 22-25, pp. 89-95, 1995. M. Vuskovic, J. Schmit, B. Dundon and C. Konopka, "Hierachical Discrimination of Grasp Modes Using Surface EMGs," Internat. IEEE Conference on Robotics and Automation, Minneapolis, Minnesota, pp. 2477-2483, April 22-28 1996.

where a # is shorthand notation for a aT . After substituting (16) into (18) and after breaking the sum into two parts we get: # n 1 S (n +m ) = x (k ) c n (x (n +1) c n )) + ( n + m 1 k =1
( ) ( )


n +m 1 (x (n +1) c n ) , n + m 1 k =n +1 (19) n n (k ) Since k =1 (x c ) = 0 , and after substituting (12),

( )


( )


the first part of (19) can be rewritten as: # n 1 n S n + 2 (x (n +1) c n ) (20) n + m 1 n + m 1 After factoring, the right part of (19) becomes: # m (21) x (n +1) c n ) . (1 ) ( n + m 1 Putting (20) and (21) together, (19) becomes: # n 1 n S (n +m ) = S n + 2 x (n +1) c n ) ( n + m 1 n + m 1 (22) If we express m in terms of and n, based on the definition of in (15), the equation (22) becomes:
( ) ( )
( ) ( ) ( )




S (n +m ) = 1 S

(n )

+ 2 (x (n +1) c n

( )

)(x (n +1) c n )
( )


(23) where:
n 1 n , 2 = (1 2 ) . n 1 n 1 If we suppose large clusters, i.e. n ! 1 , coefficients i can be simplified:
1 = (1 )


1 (1 ),

2 (1 2 ) .


[17] [1] G.A. Carpenter and S. Grossberg, A massively parallel architecture for a self-organizing neural pattern recognition machine," Computer Vision, Graphics, and Image Processing, 37:54-115, 1987. G.A. Carpenter, Neural network models for pattern recognition and associative memory," Neural Networks, 2:243-257, 1989. G.A. Carpenter, S. Grossberg, and J.H. Reynolds, ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network, Neural Networks, 4:565-588, 1991. G.A. Carpenter, S. Grossberg, and D.B. Rosen, "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system," Neural Networks, 4, 759-771, 1991. [18]