Introduction

Association rule mining aims in extracting interesting correlations or patterns among sets of
items in the transaction databases or other data repositories. It is one of the most important and
well researched techniques of data mining. The application areas of data mining varies from
market analysis to business strategies has now extended to medical domain, temporal data,
spatial data and web applications. Hence the accuracy of the rules mined and the relationship
between attributes has become an important issue.
The typical association rule mining algorithm is decomposed into two sub problems. One is to
find those itemsets whose occurrences exceed a predefined threshold in the database; those
itemsets are called frequent or large itemsets. The other problem is to generate association rules
from those large itemsets with the constraints of minimal confidence. The rules generated are
with single objective aiming at accuracy alone. Pruning and summarization are needed to filter
the significant rules. Combine two paragraphs into single Para
The efficiency of association rule mining could be enhanced by
Reducing the number of passes over the database
Making the process as multiobjective process
Parallelization
Evolutionary algorithms (EA) such as Genetic Algorithm (GA) and Particle Swarm Optimization
(PSO), are the solution for meeting the above three requirements. Both methods have proven to
generate association rules with better predictive accuracy in reduced execution time. The issue
with Genetic Algorithm is its steady progress towards the global optima consuming much time
and that of Particle Swarm Optimization is its premature convergence.
Existing methodology in few lines
In solving problems the properties of GA and PSO affect the performance. The properties of both
depend on the parameter setting and hence user needs to tune the parameters to optimize the
performance. Setting robust values for the control parameters is not a trivial task. The interaction
between the parameters with the GA and PSO performance is a complex relationship. Without
prior knowledge of the problem, parameter tuning is very difficult and time consuming. Constant
and fixed values for the control parameters do not lead to optimal performance in many
problems. Different problems usually require different values for these parameters, which have a
significant impact on performance. The search space must be widely explored before
concentrating in the most interesting areas. Otherwise the static parameters might lead to suboptimal searches. Parameters usually interact in a complex way, so a single parameter will have a
different effect depending on the value of the others. Reduce this paragraph
Different combinations of parameter values are to be tried to find the best setting. The two major
forms of setting parameter values are through parameter tuning and parameter control.
By parameter tuning we mean that to find appropriate values for the parameters before
the execution the algorithm. The parameter values those are optimal at the beginning of
evolution becomes unsuitable during the evolutionary process.
1

Parameter control forms an alternative. Steady modification of the control parameters

during run is needed. The automatic adjustment of these parameters is therefore the
solution. Adaption of the parameters during the evolution is an effort in this direction.

Existing methods with need for the tuning all the parameters
This paper proposes methodology for mining association rules using GA and PSO through
adaptive mechanism adjusting the control parameters involved. The rest of the paper is organized
as follows.
2. Preliminaries
2.1

Association Rule

Association rules are a class of important regularities in data. Association rule mining is
commonly stated as: Let I = {i1,i2, , in} be a set of n binary attributes called items. Let D = {t1,
t2, , tm} be a set of transactions called the database. Each transaction in D has a unique
transaction ID and contains a subset of the items in I. A rule is defined as an implication of the
form X Y where X, Y I and X Y = . The sets of itemsets X and Y are called antecedent
(left-hand-side or LHS) and consequent (right-hand-side or RHS) of the rule. Often rules are
restricted to only a single item in the consequent.
There are two important basic measures for association rules, support and confidence. Support of
an association rule is defined as the percentage/fraction of records that contain X Y to the total
number of records in the database. The count for each item is increased by one every time the
item is encountered in different transaction T in database D during the scanning process. Support
is calculated using the following equation
(1)
Confidence of an association rule is defined as the percentage/fraction of the number of
transactions that contain X Y to the total number of records that contain X, where if the
percentage exceeds the threshold of confidence an interesting association rule XY can be
generated.
(

( |

(2)

Confidence is a measure of strength of the association rule

2.2 Multiobjective Optimization
A general minimization problem of M objectives can be mathematically stated as: Given
= [x1, x2, . . . , xd], where d is the dimension of the decision variable space,
Minimize : (

[ (

], subject to :
2

gj( ) 0, j= 1, 2, . . . , J, and
hk( ) = 0, k = 1, 2, . . .,K, where ( is the ith objective function, gj( ) is the jth inequality
constraint, and hk( ) is the kth equality constraint.
A solution is said to dominate another solution if it is not worse than that solution in all the
objectives and is strictly better than that in at least one objective. The solutions over the entire
solution space that are not dominated by any other solution are called Pareto-optimal solutions.
Association rule mining using GA and PSO is treated as a multiobjective problem where the
objectives are to increase the following attributes
Predictive Accuracy
Interestingness Measure
Laplace
Predictive Accuracy measures the effectiveness of the rules mined. The mined rules must have
high predictive accuracy.
(

(3)

| |

where |X&Y| is the number of records that satisfy both the antecedent X and consequent Y, |X|
is the number of rules satisfying the antecedent X.
Interestingness Measure for a rule is taken from relative confidence and is as follows
(

(
(

(
(

(4)

Laplace is a confidence estimator that takes support into account, becoming more pessimistic as
the support of X decreases.
(

(
(

(5)

2.3 Genetic Algorithm

Genetic algorithm (GA) is a search method for solving optimization problems and modeling
evolutionary systems. A typical GA works with a population of individuals, each being a
solution of the optimization problem with an associated fitness value. Successive generations of
the population are produced based on the principle of survival of the fittest. The evolution
process is achieved through genetic operations, such as crossover and mutation, on the
population of individuals. The algorithm of the Genetic Algorithm is as given below.
Genetic algorithm ( )
{
Initialize population randomly;
Evaluate fitness of each individual in the population;
While stopping condition not achieved
3

{
Perform selection;
Perform crossover and mutation;
Evaluate fitness of each individual in the population;
}
}
In traditional GA, the three basic operators namely, selection, crossover and mutation operators
are fixed apriori. The speed and success of the GA depends greatly on the correct mix of GA
operators and the probabilities of crossover and mutation operations. The optimum parameters
for these operators depend on problem on which the GA is applied and also on the fitness of the
current population.
2.4 Particle Swarm Optimization
Swarm Intelligence (SI) is an innovative distributed intelligent paradigm for solving optimization
problems that originally took its inspiration from the biological examples by swarming, flocking
and herding phenomena in vertebrates. Particle Swarm Optimization (PSO) incorporates
swarming behaviors observed in flocks of birds, schools of fish, or swarms of bees, and even
human social behavior.
PSO is initialized with a group of random particles (solutions) and then searches for optima by
updating generations. During all iterations, each particle is updated by following the two best
values. The first one is the best solution (fitness) it has achieved so far. The fitness value is also
stored. This value is called pBest. The other best value that is tracked by the particle swarm
optimizer is the best value obtained so far by any particle in the population. This best value is a
global best and is called gBest. After finding the two best values, each particle updates its
corresponding velocity and position.
Each particle p, at some iteration t, has a position x (t), and a displacement velocity v(t). The
particles best (pBest) and global best (gBest) position are stored in the associated memory. The
velocity and position are updated using equations 1 and 2 respectively.
( (

( (

(6)
(7)

Where
is the inertia weight
vi
is the particle velocity of the ith particle
xi
is the ith, or current, particle
i
is the particles number
d
is the dimension of searching space
rand ( ) is a random number in (0, 1)
c1
is the individual factor
c2
is the societal factor
pBest is the particle best
4

gBest is the global best

Inertia weight controls the impact of the velocity history into the new velocity and balances
global and local searches. Suitable fine-tuning of cognitive and social parameters c1 and c2
result in faster convergence of the algorithm and alleviate the risk of settling in one of the local
minima.
The Pseudo code for PSO is given below.
For each particle
Initialize particle
END
Do
For each particle
Calculate fitness value
If the fitness value is better than the best fitness value (pBest) in history
set current value as the new pBest
End
Choose the particle with the best fitness value of all the particles as the gBest
For each particle
Calculate particle velocity according equation (a)
Update particle position according equation (b)
End
While maximum iterations or minimum error criteria is not attained
3. Literature review

4. Methodology
Any heuristic search can be characterized by two concepts, exploitation and exploration. These
concepts are often conflicting in the sense that if exploitation of a search is increased, then
exploration decreases and vice versa. The manipulation of control parameters in GA and PSO
balances the above problem
4.1 Adaptive Genetic Algorithm
Genetic algorithms when applied for mining association rules performs global search and copes
better with attribute interaction. With roulette wheel selection the parents for crossover and
mutations are selected based on their fitness, i.e. if a candidate is having more fitness function
value then the chance of getting selected is more. The crossover and mutation operations are
manifested by the crossover and mutation rate defined by the user. The efficiency of the rules
mined by genetic algorithm mainly depends on the mutation rate. Therefore tuning the value for
the mutation rate becomes an important criterion for mining association rues with GA. Higher
5

mutation rate results in generation of chromosomes much deviated from original values thereby
resulting in higher exploration time. When mutation rates are set low then the chromosomes
reproduced are too crowded towards global optima region thereby limiting the search space.
Hence setting the mutation rate adaptively where its changes from generation to generation based
on the feedback is the solution.
The algorithm for adaptive genetic algorithm for mining association rules is as given below
{

Initialize population randomly;

Evaluate fitness of each individual in the population;
While stopping condition not achieved
{
Perform selection;
Perform crossover and mutation;
Evaluate fitness of each individual;
Change mutation operator.
}
}
The mutation operator is made adaptive as given in equation.
(

(
(

(8)

pm n is the nth generation mutation rate, pm(n+1) is the (n+1)th generation mutation rate. The first
generation mutation rate is pm0, fi(m) is the fitness of the nth individual itemset i. fmax(n+1) is the
highest fitness of the (n+1)th individual stocks. fin is the fitness of the nth individual i. m is the
number of item sets. is the adjustment factor, which is set within range [0,1].
4.2 Adaptive Particle swarm Optimization (APSO)
A swarm consists of a set of an integer number, M, of particles, xi, moving within the search
space, S Rd, each representing a potential solution of the problem
Find;

subject to appropriate constraints;

where F is the fitness function associated with the problem, which we consider to be a
minimization problem without loss of generality.
PSO is mainly conducted by three key parameters: the inertia weight and two positive
acceleration coefficients. Inertia weight controls the impact of the velocity history into the new
velocity. Acceleration parameters c1 and c2 are typically two positive constants, called the
cognitive and social parameters, respectively.

The role of the inertia weight is considered critical for the PSO algorithms convergence
behavior. As it balances global and local searches, it has been suggested to have it decrease
linearly with time, usually in away to first emphasize global search Suitable fine-tuning of
cognitive and social parameters c1 and c2 in may result in faster convergence of the algorithm
and alleviate the risk of settling in one of the local minima. The pseudo code for the Adaptive
PSO is given below
/* Ns: size of the swarm, C: maximum number of iterations, Of : the final output */
1) t = 0, randomly initialize S0,
Initialize xi, i, i {1, . . .,Ns}
Initialize vi, i, i {1, . . .,Ns}
Pbi xi, i, i {1, . . .,Ns}
Gb xi

/* xi : the ith particle */

/* vi : the velocity of the ith particle*/
/* Pbi : the personal best of the ith particle */
/* Gb : the global best particle */

2) for t = 1 to t = C,
for i = 1 to i = Ns
f(xi) = conf(xi ) log (sup (xi) (length(x) + 1)
If ( f(xi) < f(Pbi)
Pbi xi
Gb min(Pb1, Pb2, , PbN)
adjust parameters(i, c1i, c2i)
vi(t) = ivi (t-1)+ c1ir1(Pbi xi) + c2i r2(Gb xi )
xi (t)= xi(t-1) + vi
At non dominated(St At)

/* Update particle best */

/* Update global best */
/* Adaptive Adjustment */
/* Velocity Updation */
/* Position Updation */
/* Updating the Archive*/

3) Of At and stop

/* Of : Output*/

/* f(xi) : Fitness of xi */

adjust parameters(i, c1i, c2i) in the above pseudo code is achieved through adaptive
mechanism is proposed here. The proposed approach distributes the evolution into four states
based on evolutionary state estimation: Convergence, Exploration, Exploitation and Jumping out.
Estimation of Evolutionary State (EES)
Based on the search behaviors and the population distribution characteristics of the PSO the EES
is done as follows
1. The distance between particles is calculated using the Euclidean distance measure for
each particle i using the equation

(9)

where N is the population size, xi and xj are the ith an jth particle in the population
respectively.
2. Calculate the evolutionary state estimator e, defines as
7

(10)
Where
is the distance measure of the gBest particle,
,
are the maximum
and minimum distance measures respectively from step1.
3. Record the evolutionary e factor for 100 generations for each dataset individually
4. Classify the estimator e into the states: Exploration, Exploitation, Convergence, Jumping
out for each dataset individually based on the evolutionary states through fuzzy
classification techniques.
5. The intervals arrived through fuzzification is as shown in table below
States/Datasets
Convergence
Exploitation
Exploration
Jumping out

Lenses
0.0, .03
0.1, 0.4
0.2. 0.7
0.6, 1

Car
Evaluation
0.0, 0.15
0.15, 0.25
0.1, 0.3
0.3, 1

Habermans
Survival
0.0, 0.4
0.3, 0.7
0.6, 0.9
0.8, 1

Postoperative
Patient
0, 0.5
0.2, 0.6
0.4, 0.8
0.7, 1

The change of state reflected as per the PSO sequence is Convergence

Jumping Out Convergence

Zoo
0.0, 0.15
0.1, 0.3
0.2, 0.4
0.3, 1
Exploitation

Adaptive Control of Acceleration Coefficients

The acceleration coefficients are made adaptive through the estimation of evolutionary states.
Parameter c1 represents the self-cognition that pulls the particle to its own historical best
position, helping explore local niches and maintaining the diversity of the swarm. Parameter c2
represents the social influence that pushes the swarm to converge to the current globally best
region, helping with fast convergence. Both the coefficients are initialized to 2 at the start of
evolution.
The strategy to adopt for the four states is as given in table.
Control Strategies of c1 and c2
State/Acceleration
Coefficient
Exploration
Exploitation
Convergence
Jumping out

c1

c2

Increase
Increase Slightly
Increase Slightly
Decrease

Decrease
Decrease Slightly
Increase Slightly
Increase

Exploration: During exploration particles should be allowed to as many optimal regions as

possible. This avoids crowding over single optima, probably local optima and explores the target
thoroughly. Increase in the value of c1 and decrease in c2 values facilitate this process.

Exploitation: In this state based on the historical best positions of each particle they group
towards the points. The local information of the particles aids this process. A slight increase in c1
advances the search around particle best (pBest) positions. At the same time the slight decrease
in c2 avoids the deception of local optima and the final global position has yet to be explored.
Convergence: In this state the swarm identifies the global optima. All the other particles in the
swarm should be lead towards the global optima region. The slight increase in the value of c2
helps this process. To fasten up the process of convergence a slight increase in the value of c1 is
adopted.
Jumping Out: The global best (gBest) particle moves away from the local optima towards
global optima, taking it away from the crowding cluster. Once any particle in the swarm reaches
this region then all particles are to follow the same pattern rapidly. A large c2 along with a
relatively small c1 value helps to obtain this goal.
The adjustments on the acceleration coefficients should be minimal. Hence, the maximum
increment or decrement between two generations is bounded by the range of [0.05,0.1]. The sum
of the acceleration coefficients is limited to 4.0, When the sum exceeds this limit then both c1 and
c2 are normalized based on equation.
(11)
Inertia Weight Adaptation
The inertia weight controls the impact of previous flying experience, which is utilized to keep the
balance between exploration and exploitation. The particle adjusts its trajectory according to its
best experience and enjoys the information of its neighbors. In addition, the inertia weight is also
an important convergence factor; the smaller the inertia weight, the faster the convergence of
PSO. A linear decrease in inertia weight gradually may deviate the particle from their global
optima. Hence a nonlinear adaptation of inertia weight as proposed in the given equation is the
solution. The global best particle is derived based on the fitness value of the particles in the
swarm. The proposed methodology for adopting the inertia weight is based on the fitness values
exhibited by the particles.

(
{
(12)