Table of Contents

Keynotes:
Reflex Delay and Its Estimation by Analyzing Brain Waves
Goutam Chakraborty
Iwate Prefectural University, Japan

New Paradigm of Mixed Signal LSI based on Time-domain Circuits for Extremely-scaled Technologies
Kunihiro Asada
University of Tokyo, Japan

Quality Assurance for UML Modeling

Michel Chaudron
LIACS, Leiden University, The Netherlands

Research papers:
8
Study on Associative Classification Model
Dong Gyu Lee, In Bae Oh, Keun Ho Ryu

Subspace Projection based Extraction of Load Shape Factors

Minghao Piao, Jong Yun Lee, Keun Ho Ryu 12

Derivation of Classification Rules from Incremental Decision Tree Committee without Feature
Redundancy
16
YongjunPiao, MinghaoPiao, Ho Sun Shon, Keun Ho Ryu

A Survey on Locality-Sensitive Hashing for Image Search

Tie Hua Zhou, Ling Wang, Cheng Hao Jin, Kyung-Ah Kim, and Keun Ho Ryu 21

An Ultra Low Cost Digital Communications Laboratory for Education

Byambajav.D, Zolboo.B, Battsetseg.B, Ariuntungalag.M 24

Design of Small Antenna for Binaural Hearing-Aids

Byambaakhuu Batnairamdal , Changyul Cheon 30

Spectrum Analyzer Based on Spartan-3E Starter Kit

Byambajav.D, Sarangoo.S 34

FPGA Based Analog Communication Systems for Education

Byambajav.D, Sumiyakhand.D, Udval.Ts, Munkhjargal.G 38

A Microsoft Kinect based virtual rehabilitation system

Erdenetsogt Davaasambuu, Chia-Chi Chiang,John Y.Chiang, Yung-Fu Chen, Sukhbaatar Bilgee 44

SVM based classifier for hot spots identification

Peipei Li, In Bae Oh, Keun Ho Ryu 51
Long-Term Mortality Prediction of Replace Patients in Acute Myocardial Infarction
Kwang Sun Ryu,Soo Ho Park, Ho Sun Shon, Jang Hwan Bae, Keun Ho Ryu 54

The energy expenditure monitoring system of trail hiking using smart phones
Mungunbagana Batzorig, Yung-Fu Chen, Hsuan-Hung Lin, John Y. Chiang, Tan-Hsu Tan, Erdenetsogt
56
Davaasambuu, Sukhbaatar Bilgee,

Mongolian Trigram Part-of-Speech Tagger

Zoljargal Munkhjargal, Purev Jaimai 63

Mongolian Named Entity Recognition

Davaadulam Davaakhuu 67

Comparison of Different Classification Techniques for Arrhythmia

Erdenetuya Namsrai, Meijing Li, Oyun-Erdene Namsrai, Taewook Kim, Keun Ho Ryu 72

Risk Factors According to gender and age in Patients with Non ST Elevation Myocardial Infarction
from KAMIR
76
Ho Sun Shon, Jong Yun Lee, Soo Ho Park, Keun Ho Ryu

A New Time Series Pattern Prediction Framework with SOM Clustering

Cheng Hao Jin, GoucholPok, Hyun-Woo Park, KwangDeuk Kim, Keun Ho Ryu 80

A Study on Utilization of Feature-based Seamless Data Model in Geospatial Information Service Aspect
Hyeongsoo Kim, Hyun Woo Park, Keun Ho Ryu 82

Nano Transistors Simulation

Narangerel N, Bolormaa D 84

Assessment of the Prediction Capability of Healthcare System Using Carotid Image and HRV analysis
HeonGyu Lee, Jin Hyoung Park, Keun Ho Ryu 87

Velocity-type respiratory air flow transducer applicable to cardiopulmonary resuscitation

In-Kwang Lee, You-Mi Lee, Keun-Ho Ryu, Suk-Kyung Shin, Mu-Song Yeu, Tae-Soo Lee, Eun-Jong Cha,
92
Kyung-Ah Kim

Comparison of Classifiers for Disease Named Entity Recognition

Tsendsuren Munkhdalai, Meijing Li, Erdenetyua.N, Oyun-Erdene.N, Taewook Kim, Keun Ho Ryu 96

Proportional-Integral control of a Self Balancing Transport

Tulga Galbadrakh, Lodoiravsal Choimaa 99

Object Classification from Video

Erdenebileg Batbaatar, Jeapil Ko, Suvdaa Batsuuri 104

Movement of 3D model, using registered motion by Microsoft Kinect motion sensing

Baatarbileg Altangerel, Erdenebat Budsuren, Oyun-erdene Namsrai
109
Video Dehazing Using Improved Dark Channel Prior
Enkhmandakh Dorj, Suvdaa Batsuuri 115

Mongolian Handwritten Recognition using Artificial Neural Network

120
Munkhzorig Bayartsogt, Lodoiravsal Choimaa, Suvdaa Batsuuri

Implementation of Hand Gesture Recognition Using TLD and Neural Networks

A.Amarsanaa, B.Suvdaa, J. Ko 124

Character Recognition System of Printed Traditional Mongolian Script

Lodoiravsal Choimaa, Batbayar Unursaikhan, Amartuvshin Renchin, Suvdaa Batsuuri 128

Comparison of IT Curriculum of Higher Education in Mongolia and the Information Technology

Engineers Examination (ITEE) Syllabus
Naranchimeg Bold, Batnyam Battulga 131

Implementation of Min-Sum Decoder for QC-LDPC Codes

Otgontsetseg Sukhbaatar, Battogtokh Jigjidsuren, Bold Zagd 136

Improving English-Mongolian statistical machine translation

Altangerel Chagnaa and Purev Jaimai 142

The Survey on Online Service at Metropolis State

Zolboo Damiran, Ganbat Tsend 145

Baseline Analysis on Current Health Statistics Information System of Mongolia

UyangaSambuu 151

Searching for Frequent XML Query Pattern Mining based on Weigted-FP-Tree

Mi Sug Gu, Keun Ho Ryu 154

Encrypted Network Based on NetFPGA

Oyunomin.B, Ankhzaya.J, Tegshjargal.J, Sodbileg.Sh 158

Development of a Mobile Tourism System

Thi Hong Nhan Vu, Yang Koo Lee, Duy Bui The 162

Trajectory Classification Using Spatiotemporal Sequence

Thi Hong Nhan Vu, Yang Koo Lee 168

Mongolian Orthography Analysis Based on Electron Text

Enkhjargal Mendsaikhan and Purev J 172

Design and Technology of Mongolian Antique findings

Gantuya Perenlei and Bold Zagd 176

Intrusion Detection System using data mining

Zolzaya Kherlenchimeg, Nyamjav Jambaljav 181

Construction of OFDM Pulse-Shapes Based on Rational Gabor Theory

Ganbold Davaajav, Bayarpurev Mongolyn 186
FPGA Implementation of Fixed-Point Box-Muller Transform
M.Dorjjamts, R.Amartuvshin, and M.Bayarpurev 191

A Gearbox Fault Detection Method under Varying Rotate Speed Conditions Based on Auxiliary
Particle Filter
195
Pan Hong-xia, Yuan Ku-mei

Launching VoIP to the Internet Communications system at the NUM

JamiyanSukhbaatar, Batpurev Mongol 201

Real and Virtual Object Integration

Nominchimeg Altansukh, Jeapil Ko, Suvdaa Batsuuri 206

3D Reconstruction from Video

Tsetsegjargal Erdenebaatar, Jaepil Ko, Suvdaa Batsuuri 207

Analysis and the current situation of Information Communication Technology (ICT) in Mongolia
Naranchimeg Bold, Batnyam Battulga 208

Critical Node Detection Problems

AltannarChinchuluun 209

A Survey on Information Communication Technology (ICT) Cluster in Mongolia

Naranchimeg Bold, Batnyam Battulga 210

Geospatial Data Indexing for University Management System using Rraster Storage
Bulganchimeg Enkhtaivan, Oyun-Erdene Namsrai 211
 The 5th International Conference FITAT 2012

Derivation of Classification Rules from Incremental Decision Tree Committee

without Feature Redundancy

Yongjun Piao, Minghao Piao, Ho Sun Shon, Keun Ho Ryu

Database/Bioinformatics Laboratory, School of Electrical & Computer Engineering,
Chungbuk National University, Korea
{ pyz, bluemhp, shon0621, khryu }@dblab.chungbuk.ac.kr

Abstract However, decision tree methods have the so-called

Previous studies show that using cascading and
sharing ensemble method can solve the singleton
coverage constraint and fragmentation problem and
derive significant classification rules. However, during
the ensemble construction, the feature redundancy is
always not concerned. Along with irrelevant features,
redundant features also affect the speed and the
accuracy of classifiers. In this paper, we propose an
efficient incremental decision tree ensemble method by
removing the feature redundancy. During the
redundancy analysis, unlike other methods, we split the
redundant features into several parts based on the
information theoretical concept of entropy. After
redundancy analysis, a number of tree committees are
constructed from each subset of features based on
cascading and sharing ensemble method. From the
tree committees, strong rules are selected for final
classification. The proposed method is compared with
three popular classifiers and all the experimental
results show that our method works well.
1. Introduction
Decision trees are used in many areas such as
artificial intelligence, machine learning, pattern
recognition and statistical analysis for gaining
information for the purpose of decision making.
Decision trees are attractive for three reasons. First,
decision trees are able to generate understandable rules.
Second, decision trees perform classification without
requiring much computation. Third, decision trees
provide a clear indication of which fields are most
important for prediction or classification. One of the
decision tree induction algorithms is Hunt‘s algorithm,
which is the basis of many existing decision tree
induction algorithm such as ID3 [1], C4.5 [2], C5.0 [3].
fragmentation problem [4]: as less and less training
data are used to search for the root nodes of sub-trees.
In order to improve the accuracy of the classifiers,
many ensemble methods have been developed.
Ensemble combines multiple base classifiers to learn a
target function and gathers their results together. It has
ability to increase accuracy by combining output of
multiple experts to reduce bias and variance, improve
efficiency by decomposing complex problem into
multiple sub problems and improve reliability by
reducing uncertainty [5]. Bagging [6] and boosting [7]
are the first approach for constructing multiple trees.
They resample the original data to create multiple
training sets based on some sampling distribution and
build the basic classifiers from each bootstrap samples.
Two interesting observations are described in [8] based
on the study of many ensemble methods. First, many
ensembles constructed by the boosting method were
singletons. Second, many top-ranked features possess
similar discriminating merits with little difference for
classification. It means that it is worthwhile to employ
different top-ranked features as the root nodes for
building multiple decision trees.
Based on these observations, in our previous work
[9], we have proposed an algorithm that constructs
incremental decision tree ensemble. However, the
growing dimensionality of recorded data demands
dimension reduction techniques that identify small sets
of features leading to a better learning performance.
Although there are many feature ranking methods
which try to select the discriminative features have
been developed, the correlation among the features was
always ignored during the ensemble construction.
Empirical evidence from feature selection literature
shows that, along with irrelevant features, redundant
features also affect the speed and accuracy of mining
algorithms and thus should be eliminated as well [10].
To increase accuracy, each classifier in ensemble

16
 The 5th International Conference FITAT 2012
should be diverse or unique in order to reduce total
error [11], but if the feature sets provide similar
information, the combination of the classifiers trained
from them will make the same error and there is no
possibility of compensation.
In this study, we propose an efficient incremental
decision tree ensemble method by removing the
redundancy of the features. During the redundancy
analysis, unlike other methods, we split the redundant
features into several parts based on the information
theory. Because, between the two redundant features,
we find that the feature which contains more
information to the target does not always lead to high
prediction accuracy than the feature that has less
information. Furthermore, the features highly
correlated with the class are also highly correlated with
each other. For each subset of features, we construct a
tree committee by using the cascading and sharing
ensemble method to break the constraint of singleton
classification rules.
The rest of this paper is organized as follows. In
section 2, the state of the art of decision tree, ensemble
method and redundancy analysis is briefly presented.
Section 3 introduces a framework of the incremental
decision tree ensemble method. Experimental results of
the proposed method conducted on four datasets are
shown in section 4. Finally, conclusions are given in
the last section.
2. Related work
2.1. Decision tree
Decision tree induction task could be categorized
into two types; one is non-incremental decision tree
induction, and the other is incremental decision tree
induction. Incremental decision tree algorithm is a tree
reconstruction method when new training instances
become available. ID4 [12] applies the ID3 to an
incremental manner to allow objects to be presented at
a time. ID5 expanded ID4 by selecting the most
suitable attribute for a node, while a new instance is
processed, restructuring the tree, so that this attribute is
pulled-up from the leaves towards that node. This is
achieved by suitable tree manipulations that allow the
counters to be recalculated without examining the past
instances. ID5R [13] is a successor of the ID5
algorithm. When we have to change the test attribute at
a decision node, instead of discarding the sub-trees,
ID5R uses a pull-up process to restructure the tree and
retains the training instances in the tree. This pull-up
process only recalculates the positive and the negative
counts of training instances during the manipulation.
17
2.2. Ensemble method
An ensemble method [14] constructs a set of base
classifiers from training data and performs
classification by taking a vote on the predictions made
by each base classifier. Bagging is a method for
generating multiple decision trees and using these trees
to get an aggregated predictor. The multiple decision
trees are formed by bootstrap aggregating which
repeatedly samples from a data set and the sampling is
done with replacement. Unlike bagging, boosting
assigns a weight to each training example and may
adaptively change the weight at the end of each
boosting round. However, bagging and boosting are
difficult to be used in incremental induction tree
process because of the expensive cost and manipulation
of the training data.
2.3. Feature ranking
High-dimensional data poses a severe challenge for
data mining. Feature weighting algorithms assign
weights to features individually and rank them based on
their relevance to the target concept. There are many
feature ranking algorithms that have been developed
such as chi-square test, mutual information, Pearson
correlation coefficients, Relief and so on [15]. These
methods are fast but lack of robustness against
interactions among features. In [10], they proposed a
Fast Correlation-Based Filter (FCBF) approach to
remove the redundant features as well as irrelevant
features. Symmetrical Uncertainty (SU) was used to
measure the correlation.
(1)
(2)
where is the information gain of X after
observing variable Y. and are the entropy
of variable X and Y individually. FCBF removes
irrelevant features by ranking correlation between
feature and class. Redundant features could be defined
from meaning of predominant feature and approximate
Markov Blanket. A feature is predominant if it does not
have any approximate Markov Blanket in the current
set. For two relevant feature Fi and Fj ( ), Fj forms
an approximate Markov Blanket for Fi if
(3)
 The 5th International Conference FITAT 2012

where is the correlation between feature and the redundancy analysis (line 9) and repeat the procedure
class. is the correlation between feature and until there are no features in Drem (line 4-15). Drem is
feature. the subset of features which are removed in each
redundancy analysis step (line 13). At first iteration, as
3. Methodology there are no features in the Drem, so we choose the most
relevant feature as the starting point (line 6-7). The
pseudo code of our algorithm is shown in Figure 2. As
Feature ranking a result, we can obtain a number of feature subsets,
then in a cascading manner, each top ranked feature
(line 17) in each subset uses as the root node of the ith
tree (Tree committees1 committees2, committeesk).
Subset1 Subset2 … Subsetk Tree induction is done by the skeleton of ITI
incremental learning algorithm [16]. The tree
committees are constructed at the point when there are
test examples or unknown instances that need to assign
Top K ranked Top K ranked Top K ranked
feature feature feature a class label. Because we found that using incremental
manner to construct the tree committees was just
wasting time and storage. Subsequently, from the tree
committees, we share the rules and the aggregate score
Tree Tree Tree
[9] is used to perform the prediction task (line 20-21).
committees1 committees2 committeesk
The classification score for a specific class, say class C,
is calculated as:
(4)

Sharing rules where, Kc denotes the number of rules in the class C,

c
and rule i denotes ith rules in class C. If the score for
Perform Classification one class C is larger than other classes, then the class
label for the instance T is assigned as C.
In this section, we introduce the proposed
incremental decision tree ensemble method which is input : D // a training data set
depicted in Fig. 1. output : Classfication rules

Figure 1. Framework of tree construction 1 begin

2 i = 1, Drem = NULL
In the feature ranking step, all the features are 3 Drel = getOrderedRelevanceSubset(D)
ranked by the symmetrical uncertainty value and a 4 do begin
relevant feature subset is selected. Suppose a dataset D 5 Dtemp = Drel
6 if ( Drem == NULL)
contains N features (F1, F2, FN) and a class C.
7 spoint = getFirstElement(Dtemp)
According to the correlation between features and class, 8 else
, all the feature are sorted in descending order. 9 spoint = getFirstElement(Drem)
Then a subset Drel of relevant features can be derived 10 //remove features whose ranking is higher
by a predefined threshold (line 2-3). Then, we split 11 //than spoint
the redundant features in the subset of relevant features 12 Dtemp = removeFeatures(Dtemp)
into several parts (subset1, subset2, subsetk). During 13 subset[i], Drem = redundantAsis(Dtemp,spoint)
the redundant analysis, if we remove the redundant 14 i++
15 end until (Drem == NULL)
features as FCBF does, the selected feature subset
16 for j=1 to i do begin
cannot guarantee the best prediction for the 17 F[j] = selectTopFeatures(subset[j])
classification problem. The reason is that, as we 18 T[j] = constructIncrementalMultipleTree(F[j])
mentioned in the previous section, the features highly 19 end
correlated with class are also highly correlated with 20 Rules = sharingRules(T)
each other so that the removed feature subset may lead 21 PredictionResult = performClassfication(Rules)
to more accurate result. So in our method, we choose 21 end;
the first element of the Drem as a starting point of the Figure 2. Tree construction algorithm

18
 The 5th International Conference FITAT 2012

4. Experimental results

Figure 3. Classification accuracy

The objective of this section is to evaluate our

proposed algorithm in terms of learning accuracy. The feature subset construction, we just repeated the
datasets used in this experiment were taken from the procedure two times, because the dimensionality of the
UCI machine learning repository and the details of dataset was not too high, so the feature subset derived
each dataset are shown in Table 1. by repeating more than two times made poor prediction.
Figure 3 shows the accuracy of our method and another
Table 1. The datasets used for experiment three classification algorithms. As shown in figure 3,
Dataset Sample Features Classes the proposed algorithm can achieve competitive
Breast Cancer 569 32 2 performances on different manipulation of the example
Ecoli 336 8 8 data with different types of classification algorithms.
Heart Disease 303 13 5 Table 2 presents the average accuracy of the classifiers.
Lung Cancer 32 57 4 Table 2. Average classification accuracy
Dataset Test C4.5 Baysian AdaBoost
We used three widely used classifiers: Naïve Bayes, Breast
97.35% 95.91% 95.37% 96.16%
C4.5 and AdaBoost. In order to make the fair Cancer
comparison, for these three algorithms, we used FCBF Ecoli 81.86% 79.44% 80.54% 59.26%
algorithm to reduce the dimensionality of the dataset. Heart
63.42% 65.18% 62.68% 64.63%
The classification results were validated by the Disease
different size of training and test dataset. During the Lung
69.74% 57.39% 69.51% 55.81%
Cancer

19
 The 5th International Conference FITAT 2012
From Table 2, one can easily observe that in most cases
the prediction performance of the proposed algorithm
is better than other classifiers. However, the accuracy
of our algorithm on Heart Disease dataset is a little
lower than other classifiers. The reason is that after
redundancy analysis, the greater part of the features is
removed and only two features are used to construct the
tree committees. Consequently thus, our algorithm
cannot greatly improve the accuracy of the classifiers
for this kind of dataset. The experiment results show
that removing redundant features is a good way to
improve the accuracy of classifiers and it is worthwhile
to use different predominant features as a starting point
to remove other features.
5. Conclusions
In this paper, an efficient incremental decision tree
ensemble method is presented to deal with high
dimensional features and feature redundancy. During
the experiments, we used four datasets which are taken
from the UCI machine learning repository and we have
met our objective which is to evaluate and investigate
the performance of the classifiers by using different
size of training and test dataset. The results show that
dividing the redundant features into several parts for
ensemble construction can obtain better performance
for classification and our proposed algorithm has
higher prediction accuracy than other classification
algorithms in most cases. Furthermore, among the
mutually redundant features, it is not a best choice to
select the most relevant feature as an input feature of
the classifier. Some future works are planned to extend
the above algorithm by applying the datasets which
have high dimensionality.
6. Acknowledgement
This work was supported by the National Research
Foundation of Korea (NRF) grant funded by the Korea
government (MEST) [No. 2012-0000478]
7. References
[1] Quinlan, J.R., ―Induction of Decision Trees‖, Machine
Learning, 1986, pp. 81-106.
[2] Quinlan, J. R., ―C4.5: Programs for Machine Learning‖,
Morgan Kaufmann, San Francisco, 1993.
[3] RuleQuest Research Data Mining Tools,
http://www.rulequest.com/
20
[4] Pagallo,G. and Haussler,D., ―Boolean feature discovery
in empirical learning‖, Machine Learning, 1993, pp. 71-99.
[5] Rakkrit Duangsoithong and Terry Windeatt, ―Hybrid
Correlation and Causal Feature Selection for Ensemble
Classifiers‖, Studies in Computational Intelligence, 2011, pp.
97-115.
[6] Breiman, L., ―Bagging predictors‖, Machine Learning 24,
1996, pp. 123-140.
[7] Freund, Y. and Schapire, R.E., ―Experiments with a New
Boosting Algorithm‖, The Thirteenth International
Conference on Machine Learing, 1996, pp. 148-156.
[8] Li, J. Y., Liu, H. A., See-Kiong Ng and Limsoon Wong,
―Discovery of significant rules for classifying cancer
diagnosis data‖, Bioinformatics, 2003, pp. 93-102.
[9] Minghao Piao, Jong Bum Lee, Khalid E.K. Saeed and
Keun Ho Ryu, ―Discovery of Significant Classification Rules
from Incrementally Inducted Decision Tree Ensemble for
Diagnosis of Disease‖, ADMA, 2009, pp. 93-102.
[10] Lei Yu and Huan Liu ―A case study of incremental
concept induction‖, SIGKDD, Washington DC, USA, 2003.
[11] Duangsoithong, R. and Windeatt, T., ―Relevant and
Redundant Feature Analysis with Ensemble Classification‖,
Advances in Pattern Recognition, 2009, pp. 247-250.
[12] Schlimmer, J. C. and Fisher, D., ―Induction of Decision
Trees‖, Proceedings of the Fifth National Conference on
Artificial Intelligence, 1986, pp. 496-501.
[13] Utgoff, P. E., ―Incremental Induction of decision trees‖,
Machine Learning, pp. 161-186, 1989.
[14] Tan, P. N., Steinbach, M., Kumar, V., Introduction to
data mining, 2006, pp. 276-277.
[15] Iffat A. Gheyas and Leslie S. Smith, ―Feature subset
selection in large dimensionality domains‖, Pattern
Recognition 43, 2010, pp. 5-13.
[16] Utgoff, P. E., Berkman, N. C. and Clouse, J. A.,
―Decision Tree Induction Based on Efficient Tree
Restructuring‖, Machine Learning, 1997, pp. 5-44.