You are on page 1of 4

### Solving Nonlinear Equation Systems Using Evolutionary Algorithms

Crina Grosan

Department of Computer Science Babes¸-Bolyai University Cluj-Napoca, 3400, Romania

cgrosan@cs.ubbcluj.ro

ABSTRACT

This paper proposes a new perspective for solving systems of

nonlinear equations. A system of equations can be viewed as

a multiobjective optimization problem: every equation rep-

resents an objective function whose goal is to minimize dif-

ference between the right and left term of the corresponding

equation in the system. We used an evolutionary computa-

tion technique to solve the problem obtained by transform-

ing the system of nonlinear equations into a multiobjective

problem. Results obtained are compared with a very new

technique  and also some standard techniques used for

solving nonlinear equation systems. Empirical results illus-

trate that the proposed method is eﬃcient.

1.

INTRODUCTION

A nonlinear system of equations is deﬁned as:

f(x) =

f 1 (x)

f 2 (x)

.

.

.

f n (x)

.

x

=

(x 1 , x 2 ,

. . .

, x n ), refers to n equations and n variables;

where

f 1 ,

. . .

, f n are nonlinear functions in the space of all

real valued continuous functions on Ω =

n

i=1

[a i , b i ] n .

Some of the equations can be linear, but not all of them.

Finding a solution for a nonlinear system of equations f(x)

involves ﬁnding a solution such that every equation in the

nonlinear system is 0:

(P)

f 1 (x 1 , x 2 , ...

f 2 (x 1 , x 2 , ...

,

,

x n ) = 0

x n ) = 0

.

.

.

f n (x 1 , x 2 , ...

,

x n ) = 0

(1)

In Figure 1 the solution for a system having two nonlinear

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proﬁt or commercial advantage and that copies bear this notice and the full citation on the ﬁrst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciﬁc permission and/or a fee. GECCO ’06 Seattle, WA, USA Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...\$5.00.

Ajith Abraham

School of Computer Science and Engineering Chung-Ang University Seoul, 156-756 Korea

ajith.abraham@ieee.org

equations is depicted.

Figure 1: Example of solution in the case of a two

nonlinear equation system represented by f 1 and f 2 . There are also situations when a system of equations is

having multiple solutions. For instance, the system:

f 1 (x 1 , x 2 , x 3 , x 4 ) =

x 2 + 2x 2 2 + cos(x 3 ) x 4 = 0

1

2

f 1 (x 1 , x 2 , x 3 , x 4 ) = 3x 2 + x 2 2 + sin 2 (x 3 ) x 4 = 0

1

2

f 1 (x 1 , x 2 , x 3 , x 4 ) =

2x 2 x 2 2 cos(x 3 ) + x 4 =

1

2

0

f 1 (x 1 , x 2 , x 3 , x 4 ) = x 2 x 2

1

2 cos 2 (x 3 ) +

2

x 4 = 0

is having two solutions: (1, -1, 0, 2) and (-1, 1, 0, -2).

The

assumption is that a zero, or root, of the system exists. The

solutions we are looking for are those points (if any) that

are common to the zero contours of f i ,i = 1,

...

, n.

There are several ways to solve nonlinear equation systems

(, -, ). Probably, the most popular techniques are

the Newton type techniques. Other techniques are:

Trust-region method ;

Broyden method ;

Secant method ;

Halley method .

Newton’s method

We can approximate f using the ﬁrst order Taylor expan-

sion in a neighborhood of a point x k

n .

The Jacobian

matrix J(x k ) nxn to f(x) evaluated at x k is given by:

Then:

J =

δf

1

δx

1

...

δf 1

δxn

.

.

.

.

.

.

δf

n

δx

1

...

δf

n

δx

n

f(x k + t) = f(x k ) + J(x k )t + O(||p|| 2 ).

By setting the right side of the equation to zero and ne-

glecting terms of higher order (except the ﬁrst) (O(||p|| 2 ))

we obtain:

J(x k )t = f(x k ).

Then, the Newton algorithm is described as follows:

: Algorithm1: Newton algorithm

Set k=0

Guess an approximate solution x 0 .

Deﬁne:

u(k) = (u 1 (k), u 2 (k),

x(k + 1) = u(k)

...

, u n (k)).

Deﬁne f 0 : Ω×U (Ω andU are compact subsets of n ):

f 0 (x(k), u(k)) = u(k) f(x(k)) 2

2 .

The error function E is deﬁned as follows:

E[x t , u t ] =

t1

k=0

f 0 (x(k), u(k)).

x t = (x(1), x(2), ...

,

x(t 1), 0)

u t (u(1), u(2),

...

, u(t 1), 0).

Consider the following problem:

(P 1 )

minimize E[x t , u t ] =

subject to

x(k + 1) = u(k)

t1

k=0

f 0 (x(k), u(k))

x(0) = 0, x(t) = 0, (x 0 is known)

In the theorem illustrated by Eﬀati and Nazemi  if

there is an optimal solution for the problem P 1 such that

the value of E will be zero, then this is also a solution for

the system of equations to be solved.

The problem is transformed to a measure theory problem.

 Repeat Compute J(x k ) and f(x k ). Solve the linear system J(x k )t = −f(x k ). By solving the transformed problem u t is ﬁrst constructed. From there, x t could be obtained (see for details ). The measure theory method is improved in . The interval Set x k+1 = x k + t. [1, t] is divided into the subintervals S 1 = [1, t− 1] and S 2 = Set t = t + 1. [t − 1, t]. The problem P 1 is solved in both subintervals and

Until converged to the solution

The index k is an iteration index and x k

after k iterations.

is the vector x

The idea of the method is to start with a value which is

reasonably close to the true zero, then replace the function

by its tangent and computes the zero of this tangent. This

zero of the tangent will typically be a better approximation

to the function’s zero, and the method can be iterated.

Remarks

• 1. This algorithm is also known as Newton-Raphson method. There are also several other Newton methods.

• 2. The algorithm converges fast to the solution.

• 3. It is very important to have a good starting value (the success of the algorithm depends on this).

• 4. The Jacobian matrix is needed but in many problems analytical derivatives are unavailable.

• 5. If function evaluation is expensive, then the cost of ﬁnite-diﬀerence determination of the Jacobian can be prohibitive.

Eﬀati’s method

In , Eﬀati and Nazemi proposed a new method for

solving systems of nonlinear equations.

posed in  is presented below.

The method pro-

The following notations are used:

 x i (k + 1) = f i (x 1 (k), x 2 (k), , x n (k)); f(x k ) = (f 1 (x k ), f 2 (x k ), , f n (x k )); i = 1, n and x i : N → . If there exist a t such that x(t) = 0 then f i (x(t − 1)) = 0, i = 1, , n. This involves that x(t−1) is an exact solution

for the given system of equations.

two errors E 1 and E 2 respectively are obtained. This way,

an upper bound for the total error is found. If this upper

bound is estimated to be zero then an approximate solution

for the problem is found.

• 2. PROBLEM TRANSFORMATION

This section explains how the problem is transformed to a

multiobjective optimization problem. First, the basic deﬁn-

itions of a multiobjective optimization problem is presented

and what it denotes an optimal solution for this problem

.

Let Ω be the search space. Consider n objective functions

 f 1 , f 2 . f n , f i : Ω → , i = 1, 2, , n where Ω ⊂ m .

The multiobjective optimization problem is deﬁned as:

optimize f(x) = (f 1 (x),

...

, f n (x))

subject to

x

= (x 1 , x 2 ,

. . .

x m ) .

For deciding wether a solution is better than another so-

lution or not, the following relationship between solutions

might be used:

Deﬁnition 1. (Pareto dominance)

Consider a maximization problem. Let x, y be two decision

vectors (solutions) from Ω.

Solution x dominate y (also written as x y) if and only if

the following conditions are fulﬁlled:

 (i) f i (x) ≥ f i (y), ∀i = 1, 2, , n, (ii) ∃j ∈ {1, 2, , n}: f j (x) > f j (y).

That is, a feasible vector x is Pareto optimal if no feasi-

ble vector y can increase some criterion without causing a

simultaneous decrease in at least one other criterion.

In the literature other terms have also been used instead of

4.

EXPERIMENTS

Pareto optimal or minimal solutions, including words such

as nondominated, noninferior, eﬃcient, functional-eﬃcient

solutions.

The solution x 0 is ideal if all objectives have their opti-

mum in a common point x 0 .

Deﬁnition 2. (Pareto front)

The images of the Pareto optimum points in the criterion

space are called Pareto front.

The system of equations (P) can be transformed into a

multiobjective optimization problem. Each equation can be

considered as an objective function. The goal of this opti-

mization function is to minimize the diﬀerence (in absolute

value) between left side and right side of the equation. Since

the right term is zero, the objective function is denoted by

the absolute value of the left term.

The system (P) is then equivalent to:

This section reports the several experiments and compar-

isons which we performed. We consider the same problems

(Example 1 and Example 2 below) as the ones used by Ef-

fati and Nazemi . Parameters used by the evolutionary

approach for both examples are given in Table 1.

Table 1: Parameter setting used by the evolutionary

approach.

 Parameter Value Example 1 Example 2 Population size 250 300 Number of generations 150 200 Sigma (for mutation) 0.1 0.1 Tournament size 4 5

(P )

 minimize abs(f 1 (x 1 , x 2 , ... , x n )) 4.1 Example 1 minimize . . . abs(f 2 (x 1 , x 2 , ... , x n )) Consider the following nonlinear system: f 1 (x 1 , x 2 ) = cos(2x 1 ) − cos(2x 2 ) − 0.4 = 0 minimize abs(f n (x 1 , x 2 , ... , x n )) f 2 (x 1 , x 2 ) = 2(x 2 − x 1 ) + sin(2x 2 ) − sin(2x 1 ) − 1.2 = 0
• 3. EVOLUTIONARY NONLINEAR EQUA- TION SYSTEM

An evolutionary technique is applied to solving the mul-

tiobjective problem obtained by transforming the system of

equations. We generate some starting points (initial solu-

tions) within deﬁned domain. Then these solutions were

evolved in an iterative manner. In order to compare two

solutions we use the Pareto dominance relationship. Ge-

netic operators (such as crossover and mutation) are used.

Convex crossover and gaussian mutation are used . An

external set was used for storing all the nondominated so-

lutions found during the iteration process. Tournament se-

lection is applied. n individuals are randomly selected from

the uniﬁed set of current population and external popula-

tion. Out of these n solutions the one which dominated a

greater number of solutions is selected. If there are two or

more ’equal’ solutions then one is picked at random. At

each iteration we update this set by introducing all the non-

dominated solutions obtained at the respective step and we

are removing form the external set all solutions which will

become dominated.

The algorithm can be described as follows:

Step 1. Set t = 0.

Randomly generate population P(t).

Set EP(t) = . (EP denoted the external population.

Step 2. Repeat

Step 2.1. Evaluate P(t).

Step 2.2. Selection (P(t) (t)).

Step 2.3. Crossover.

Step 2.4. Mutation.

Step 2.3. Select all nondominated individuals

obtained.

Step 2.3. Update EP(t).

Step 2.3. Update (P(t) (keep best between

parents and oﬀspring).

Step 2.3. Set t := t + 1.

Until t = number o f g enerations.

Step 3. Print EP(t).

Results obtained by applying Newton’s method, Eﬀati’s

technique and the proposed method are presented in Table

2.

As evident from Table 2, results obtained by the Evolu-

tionary Algorithm (EA) are better than the ones obtained

by the other techniques. Also, by applying an evolutionary

technique we don’t need any additional information about

the problem (such as the functions to be diﬀerentiable, an

adequate selection of the starting point, etc).

Table 2: Results for the ﬁrst example.

 Method Solution Functions values Newton (0.15, 0.49) (-0.00168, 0.01497) Eﬀati (0.1575, 0.4970) (0.005455, 0.00739) EA (0.15772, 0.49458) (0.001264, 0.000969)
• 4.2 Example 2

We have the following problem:

f 1 (x 1 , x 2 ) = e x 1 + x 1 x 2 1 = 0

f 2 (x 1 , x 2 ) =

sin(x 1 x 2 ) + x 1 + x 2 1 = 0

Results obtained by Eﬀati’s method and the evolutionary

approach are given in Table 3. For this example, the evolu-

tionary approach obtained better results better than Eﬀati’s

method. These experiments show the eﬃciency and advan-

tage of applying evolutionary techniques for solving systems

of nonlinear equations against standard mathematical ap-

proaches.

Table 3: Results for the second example.

 Method Solution Functions values Eﬀati (0.0096, 0.9976) (0.019223, 0.016776) EA (-0.00138, 1.0027) (-0.00276, -6,37E-5)
• 5. CONCLUSIONS

The proposed approach seems to be very eﬃcient for solv-

ing equation systems. We analyzed here the case of non-

linear equation systems. The proposed approach could be

extended for higher dimensional systems. Also, in a similar

manner, we can solve inequations systems.

• 6. REFERENCES

 C. Brezinski, Projection methods for systems of

equations, Elsevier, 1997.

 C.G. Broyden, A class of methods for solving

nonlinear simultaneous equations. Mathematics of

Computation, 19, 577-593, 1965.

 A.R. Conn, N.I.M. Gould, P.L. Toint, Trust-Region

methods, SIAM, Philadelphia, 2000.

 A. Cuyt, P.van der Cruyssen, Abstract Pade

approximants for the solution of a system of nonlinear

equations, Computational Mathematics and

Applications, 9, 139-149, 1983.

 J.E. Denis, On Newtons Method and Nonlinear

Simultaneous Replacements, SIAM Journal of

Numerical Analisys, 4, 103108, 1967.

 J.E. Denis, On Newtonlike Methods, Numerical

Mathematics, 11 , 324330, 1968.

 J.E. Denis, On the Convergence of Broydens Method

for Nonlinear Systems of Equations, Mathematics of

Computation, 25, 559567, 1971.

 J.E. Denis, H. Wolkowicz, LeastChange Secant

Methods, Sizing, and Shifting, SIAM Journal of

Numerical Analisys, 30, 12911314, 1993.

 J.E. Denis, M. ElAlem, K. Williamson, A

Trust-Region Algorithm for Least-Squares Solutions of

Nonlinear Systems of Equalities and Inequalities,

SIAM Journal on Optimization 9(2), 291-315, 1999.

 S. Eﬀati, A.R. Nazemi, A new methid for solving a

system of the nonlinear equations, Applied

Mathematics and Computation, 168, 877-894, 2005

 Goldberg, D.E. Genetic algorithms in search,

optimization and machine learning. Addison Wesley,

Reading, MA, 1989.

 W. Gragg, G. Stewart, A stable variant of the secant

method for solving nonlinear equations, SIAM Journal

of Numerical Analisys, 13, 889-903, 1976.

 J. M. Ortega and W. C. Rheinboldt, Iterative solution

of nonlinear equations in several variables. New York:

Academic Press, 1970

 W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P.

Flannery, Numerical Recipes in C: The Art of

Scientiﬁc Computing, Cambridge University Press,

2002.

 Steuer, R. E. Multiple Criteria Optimization. Theory,

Computation, and Application. Wiley Series in

Probability and Mathematical Statistics: Applied

Probability and Statistics. New York: John Wiley

Sons, Inc, 1986.