3elements of the same node are partitioned andcombined by the gating network and the total outputof the node is given as a convex combination. Theexpert networks are trained to increase the posteriorprobability according to Bayes rule and then a numberof learning algorithms can be applied to tune themixture model.Recently the EM algorithm was developed for theHME architecture . The tests on the robot dynamicsproblem showed a substantial improvement incomparison with the back-propagation neural network.The HME technique does not seem to be applicablefor a large dimensional data, as increase of thecomplexity of the tree-like architecture and associatedinput space subdivision lead to the increased varianceand numerical instability.
3. FUSING SINGLE CLASS LABELS
Classifiers producing crisp, single class labels (SCL)provide the least amount of useful information for thecombination process. However, they are still wellperforming classifiers, which could be applied to avariety of real-life problems. If some training data areavailable, it is possible to upgrade the outputs of theseclassifiers to the group operating on class rankings oreven fuzzy measures. There are a number of methodsto achieve this goal, for instance by performing anempirical probability distribution over a set of trainingdata. The two most representative methods for fusingSCL classifiers, namely generalised voting method,and Knowledge-Bahaviour Space method, are nowpresented.
3.1 Voting Methods
Voting strategies can be applied to a multipleclassifier system assuming that each classifier gives asingle class label as an output and no training data areavailable. There are a number of approaches tocombination of such uncertain information units inorder to obtain the best final decision. However, theyall lead to the generalised voting definition. Forconvenience let the output of the classifiers form thedecision vector
denotes the label of the i-th classand
the rejection of assigning the input sample toany class. Let binary characteristic function be definedas follows:
i ji ji j
cdcdif if 01)(cB
Then the general voting routine can be defined as:
n1 ji jn1 jt jm,..,1t
where is a parameter and
is a function thatprovides additional voting constraints. The mostconservative voting rule is given if
,meaning that the class is chosen when all classifiersproduce the same output. This rule can be liberalisedby lowering the parameter . The case where
is commonly known as the majority vote. Function
is usually interpreted as a level of abjection tothe most often selected class and refers mainly to thescore of the second ranked class. This option allows toadjust the level of collision that is still acceptable forgiving correct decision.
3.2 Behaviour-Knowledge Space Method
Most fusion methods assume independence of thedecisions made by individual classifiers. This is in factnot necessarily true and Behaviour-Knowledge Spacemethod (BKS) does not require this condition . Itprovides a knowledge space by collecting the recordsof the decisions of all classifiers for each learnedsample. If the decision fusion problem is defined as amapping of
, the method operates on the
- dimensionalspace. Each dimension corresponds to an individualclassifier, which can produce
class labels and one rejection decision. A unit of BKS is an intersection of decisions of every singleclassifier. Each BKS unit contains three types of data:the total number of incoming samples:
, the bestrepresentative class:
, and the total number of incoming samples for each class:
. In thefirst stage of BKS method the training data areextensively exploited to build the BKS. Then the finalclassification decision for an input sample is derivedin the focal unit where the balance is estimatedbetween the current classifiers decisions and therecorded behaviour information as shown in thefollowing rule:
where is a threshold controlling the reliability of thefinal decision. The model tuning process shouldinclude automatic finding of the threshold .
4. CLASS RANKING BASED TECHNIQUES
Among the fusion methods operating on classrankings as the outputs from multiple classifiers, twomain approaches are worth mentioning. The first isbased on a class set reduction and its objective is toreduce the set of considered classes to as small anumber as possible but ensuring that the correct classis still represented in the reduced set. Another