Crina Grosan
Department of Computer Science Babes¸Bolyai University ClujNapoca, 3400, Romania
cgrosan@cs.ubbcluj.ro
ABSTRACT
This paper proposes a new perspective for solving systems of
nonlinear equations. A system of equations can be viewed as
a multiobjective optimization problem: every equation rep
resents an objective function whose goal is to minimize dif
ference between the right and left term of the corresponding
equation in the system. We used an evolutionary computa
tion technique to solve the problem obtained by transform
ing the system of nonlinear equations into a multiobjective
problem. Results obtained are compared with a very new
technique [10] and also some standard techniques used for
solving nonlinear equation systems. Empirical results illus
trate that the proposed method is eﬃcient.
1.
INTRODUCTION
A nonlinear system of equations is deﬁned as:
f(x) =
f _{1} (x)
f _{2} (x)
.
.
.
f _{n} (x)
.
x
=
(x _{1} , x _{2} ,
. . .
, x _{n} ), refers to n equations and n variables;
where
f _{1} ,
. . .
, f _{n} are nonlinear functions in the space of all
real valued continuous functions on Ω =
n
i=1
[a _{i} , b _{i} ] ⊂ ^{n} .
Some of the equations can be linear, but not all of them.
Finding a solution for a nonlinear system of equations f(x)
involves ﬁnding a solution such that every equation in the
nonlinear system is 0:
(P)
f _{1} (x _{1} , x _{2} , ...
f _{2} (x _{1} , x _{2} , ...
,
,
x _{n} ) = 0
x _{n} ) = 0
.
.
.
f _{n} (x _{1} , x _{2} , ...
,
x _{n} ) = 0
(1)
In Figure 1 the solution for a system having two nonlinear
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proﬁt or commercial advantage and that copies bear this notice and the full citation on the ﬁrst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciﬁc permission and/or a fee. GECCO ’06 Seattle, WA, USA Copyright 200X ACM XXXXXXXXX/XX/XX ...$5.00.
Ajith Abraham
School of Computer Science and Engineering ChungAng University Seoul, 156756 Korea
ajith.abraham@ieee.org
equations is depicted.
Figure 1: Example of solution in the case of a two
nonlinear equation system represented by f _{1} and f _{2} .
There are also situations when a system of equations is
having multiple solutions. For instance, the system:
f _{1} (x _{1} , x _{2} , x _{3} , x _{4} ) =
x ^{2} + 2x ^{2} _{2} + cos(x _{3} ) − x _{4} = 0
1
2
f _{1} (x _{1} , x _{2} , x _{3} , x _{4} ) = 3x ^{2} + x ^{2} _{2} + sin ^{2} (x _{3} ) − x _{4} = 0
1
2
f _{1} (x _{1} , x _{2} , x _{3} , x _{4} ) =
−2x ^{2} − x ^{2} _{2} − cos(x _{3} ) + x _{4} =
1
2
0
f _{1} (x _{1} , x _{2} , x _{3} , x _{4} ) = −x ^{2} − x ^{2}
1
_{2} − cos ^{2} (x _{3} ) +
2
x _{4} = 0
is having two solutions: (1, 1, 0, 2) and (1, 1, 0, 2).
The
assumption is that a zero, or root, of the system exists. The
solutions we are looking for are those points (if any) that
are common to the zero contours of f _{i} ,i = 1,
...
, n.
There are several ways to solve nonlinear equation systems
([1], [5][9], [13]). Probably, the most popular techniques are
the Newton type techniques. Other techniques are:
• Trustregion method [3];
• Broyden method [2];
• Secant method [12];
• Halley method [4].
Newton’s method
We can approximate f using the ﬁrst order Taylor expan
sion in a neighborhood of a point x ^{k}
∈ ^{n} .
The Jacobian
matrix J(x ^{k} ) ⊂ ^{n}^{x}^{n} to f(x) evaluated at x ^{k} is given by:
Then:
J =
δf
_{1}
δx
_{1}
...
δf _{1}
δxn
.
.
.
.
.
.
δf
_{n}
δx
_{1}
...
δf
_{n}
δx
_{n}
f(x ^{k} + t) = f(x ^{k} ) + J(x ^{k} )t + O(p ^{2} ).
By setting the right side of the equation to zero and ne
glecting terms of higher order (except the ﬁrst) (O(p ^{2} ))
we obtain:
J(x ^{k} )t = −f(x ^{k} ).
Then, the Newton algorithm is described as follows:
: Algorithm1: Newton algorithm
Set k=0
Guess an approximate solution x ^{0} .
Deﬁne:
u(k) = (u _{1} (k), u _{2} (k),
x(k + 1) = u(k)
...
, u _{n} (k)).
Deﬁne f ^{0} : Ω×U → (Ω andU are compact subsets of ^{n} ):
f ^{0} (x(k), u(k)) = u(k) − f(x(k)) ^{2}
2 .
The error function E is deﬁned as follows:
E[x ^{t} , u ^{t} ] =
t−1
k=0
f ^{0} (x(k), u(k)).
x ^{t} = (x(1), x(2), ...
,
x(t − 1), 0)
u ^{t} (u(1), u(2),
...
, u(t − 1), 0).
Consider the following problem:
(P _{1} )
minimize E[x ^{t} , u ^{t} ] =
subject to
x(k + 1) = u(k)
t−1
k=0
f ^{0} (x(k), u(k))
x(0) = 0, x(t) = 0, (x ^{0} is known)
In the theorem illustrated by Eﬀati and Nazemi [10] if
there is an optimal solution for the problem P _{1} such that
the value of E will be zero, then this is also a solution for
the system of equations to be solved.
The problem is transformed to a measure theory problem.
Repeat Compute J(x ^{k} ) and f(x ^{k} ). Solve the linear system J(x ^{k} )t = −f(x ^{k} ). 
By solving the transformed problem u ^{t} is ﬁrst constructed. From there, x ^{t} could be obtained (see for details [10]). The measure theory method is improved in [10]. The interval 

Set 
x ^{k}^{+}^{1} = x ^{k} + t. 
[1, t] is divided into the subintervals S _{1} = [1, t− 1] and S _{2} = 
Set t = t + 1. 
[t − 1, t]. The problem P _{1} is solved in both subintervals and 
Until converged to the solution
The index k is an iteration index and x ^{k}
after k iterations.
is the vector x
The idea of the method is to start with a value which is
reasonably close to the true zero, then replace the function
by its tangent and computes the zero of this tangent. This
zero of the tangent will typically be a better approximation
to the function’s zero, and the method can be iterated.
Remarks
1. This algorithm is also known as NewtonRaphson method. There are also several other Newton methods.
2. The algorithm converges fast to the solution.
3. It is very important to have a good starting value (the success of the algorithm depends on this).
4. The Jacobian matrix is needed but in many problems analytical derivatives are unavailable.
5. If function evaluation is expensive, then the cost of ﬁnitediﬀerence determination of the Jacobian can be prohibitive.
Eﬀati’s method
In [10], Eﬀati and Nazemi proposed a new method for
solving systems of nonlinear equations.
posed in [10] is presented below.
The method pro
The following notations are used:
x _{i} (k 
+ 1) = f _{i} (x _{1} (k), x _{2} (k), 
, 
x _{n} (k)); 

f(x _{k} ) 
= (f _{1} (x _{k} ), f _{2} (x _{k} ), 
, f _{n} (x _{k} )); 

i = 1, 
n and x _{i} : N → . 

If there exist a t such that x(t) = 0 then f _{i} (x(t − 1)) = 

0, i = 1, 
, n. This involves that x(t−1) is an exact solution 
for the given system of equations.
two errors E _{1} and E _{2} respectively are obtained. This way,
an upper bound for the total error is found. If this upper
bound is estimated to be zero then an approximate solution
for the problem is found.
2. PROBLEM TRANSFORMATION
This section explains how the problem is transformed to a
multiobjective optimization problem. First, the basic deﬁn
itions of a multiobjective optimization problem is presented
and what it denotes an optimal solution for this problem
[15].
Let Ω be the search space. Consider n objective functions
f _{1} , f _{2} 
. 
f _{n} , 

f _{i} : Ω 
→ , 
i = 1, 2, 
, n 

where 
Ω ⊂ ^{m} . 
The multiobjective optimization problem is deﬁned as:
optimize f(x) = (f _{1} (x),
...
, f _{n} (x))
subject to
x
= (x _{1} , x _{2} ,
. . .
x _{m} ) ∈ Ω.
For deciding wether a solution is better than another so
lution or not, the following relationship between solutions
might be used:
Deﬁnition 1. (Pareto dominance)
Consider a maximization problem. Let x, y be two decision
vectors (solutions) from Ω.
Solution x dominate y (also written as x y) if and only if
the following conditions are fulﬁlled:
(i) 
f _{i} (x) ≥ f _{i} (y), ∀i = 1, 2, 
, n, 
(ii) 
∃j ∈ {1, 2, , n}: f _{j} (x) > f _{j} (y). 
That is, a feasible vector x is Pareto optimal if no feasi
ble vector y can increase some criterion without causing a
simultaneous decrease in at least one other criterion.
In the literature other terms have also been used instead of
4.
EXPERIMENTS
Pareto optimal or minimal solutions, including words such
as nondominated, noninferior, eﬃcient, functionaleﬃcient
solutions.
The solution x ^{0} is ideal if all objectives have their opti
mum in a common point x ^{0} .
Deﬁnition 2. (Pareto front)
The images of the Pareto optimum points in the criterion
space are called Pareto front.
The system of equations (P) can be transformed into a
multiobjective optimization problem. Each equation can be
considered as an objective function. The goal of this opti
mization function is to minimize the diﬀerence (in absolute
value) between left side and right side of the equation. Since
the right term is zero, the objective function is denoted by
the absolute value of the left term.
The system (P) is then equivalent to:
This section reports the several experiments and compar
isons which we performed. We consider the same problems
(Example 1 and Example 2 below) as the ones used by Ef
fati and Nazemi [10]. Parameters used by the evolutionary
approach for both examples are given in Table 1.
Table 1: Parameter setting used by the evolutionary
approach.
Parameter 
Value 

Example 1 
Example 2 

Population size 
250 
300 
Number of generations 
150 
200 
Sigma (for mutation) 
0.1 
0.1 
Tournament size 
4 
5 
(P ^{} )
minimize 
abs(f _{1} (x _{1} , x _{2} , ... 
, 
x _{n} )) 


minimize . . . 
abs(f _{2} (x _{1} , x _{2} , ... 
, 
x _{n} )) 
Consider the following nonlinear system: 


f _{1} (x _{1} , x _{2} ) = 
cos(2x _{1} ) − cos(2x _{2} ) − 0.4 = 0 

minimize 
abs(f _{n} (x _{1} , x _{2} , ... 
, 
x _{n} )) 
f _{2} (x _{1} , x _{2} ) = 2(x _{2} − x _{1} ) + sin(2x _{2} ) − sin(2x _{1} ) − 1.2 = 0 
3. EVOLUTIONARY NONLINEAR EQUA TION SYSTEM
An evolutionary technique is applied to solving the mul
tiobjective problem obtained by transforming the system of
equations. We generate some starting points (initial solu
tions) within deﬁned domain. Then these solutions were
evolved in an iterative manner. In order to compare two
solutions we use the Pareto dominance relationship. Ge
netic operators (such as crossover and mutation) are used.
Convex crossover and gaussian mutation are used [11]. An
external set was used for storing all the nondominated so
lutions found during the iteration process. Tournament se
lection is applied. n individuals are randomly selected from
the uniﬁed set of current population and external popula
tion. Out of these n solutions the one which dominated a
greater number of solutions is selected. If there are two or
more ’equal’ solutions then one is picked at random. At
each iteration we update this set by introducing all the non
dominated solutions obtained at the respective step and we
are removing form the external set all solutions which will
become dominated.
The algorithm can be described as follows:
Step 1. Set t = 0.
Randomly generate population P(t).
Set EP(t) = ∅. (EP denoted the external population.
Step 2. Repeat
Step 2.1. Evaluate P(t).
Step 2.2. Selection (P(t) ∪ (t)).
Step 2.3. Crossover.
Step 2.4. Mutation.
Step 2.3. Select all nondominated individuals
obtained.
Step 2.3. Update EP(t).
Step 2.3. Update (P(t) (keep best between
parents and oﬀspring).
Step 2.3. Set t := t + 1.
Until t = number _{o} f _{g} enerations.
Step 3. Print EP(t).
Results obtained by applying Newton’s method, Eﬀati’s
technique and the proposed method are presented in Table
2.
As evident from Table 2, results obtained by the Evolu
tionary Algorithm (EA) are better than the ones obtained
by the other techniques. Also, by applying an evolutionary
technique we don’t need any additional information about
the problem (such as the functions to be diﬀerentiable, an
adequate selection of the starting point, etc).
Table 2: Results for the ﬁrst example.
Method 
Solution 
Functions values 
Newton 
(0.15, 0.49) 
(0.00168, 0.01497) 
Eﬀati 
(0.1575, 0.4970) 
(0.005455, 0.00739) 
EA 
(0.15772, 0.49458) 
(0.001264, 0.000969) 
4.2 Example 2
We have the following problem:
f _{1} (x _{1} , x _{2} ) = e ^{x} ^{1} + x _{1} x _{2} − 1 = 0
f _{2} (x _{1} , x _{2} ) =
sin(x _{1} x _{2} ) + x _{1} + x _{2} − 1 = 0
Results obtained by Eﬀati’s method and the evolutionary
approach are given in Table 3. For this example, the evolu
tionary approach obtained better results better than Eﬀati’s
method. These experiments show the eﬃciency and advan
tage of applying evolutionary techniques for solving systems
of nonlinear equations against standard mathematical ap
proaches.
Table 3: Results for the second example.
Method 
Solution 
Functions values 
Eﬀati 
(0.0096, 0.9976) 
(0.019223, 0.016776) 
EA 
(0.00138, 1.0027) 
(0.00276, 6,37E5) 
5. CONCLUSIONS
The proposed approach seems to be very eﬃcient for solv
ing equation systems. We analyzed here the case of non
linear equation systems. The proposed approach could be
extended for higher dimensional systems. Also, in a similar
manner, we can solve inequations systems.
6. REFERENCES
[1] C. Brezinski, Projection methods for systems of
equations, Elsevier, 1997.
[2] C.G. Broyden, A class of methods for solving
nonlinear simultaneous equations. Mathematics of
Computation, 19, 577593, 1965.
[3] A.R. Conn, N.I.M. Gould, P.L. Toint, TrustRegion
methods, SIAM, Philadelphia, 2000.
[4] A. Cuyt, P.van der Cruyssen, Abstract Pade
approximants for the solution of a system of nonlinear
equations, Computational Mathematics and
Applications, 9, 139149, 1983.
[5] J.E. Denis, On Newtons Method and Nonlinear
Simultaneous Replacements, SIAM Journal of
Numerical Analisys, 4, 103108, 1967.
[6] J.E. Denis, On Newtonlike Methods, Numerical
Mathematics, 11 , 324330, 1968.
[7] J.E. Denis, On the Convergence of Broydens Method
for Nonlinear Systems of Equations, Mathematics of
Computation, 25, 559567, 1971.
[8] J.E. Denis, H. Wolkowicz, LeastChange Secant
Methods, Sizing, and Shifting, SIAM Journal of
Numerical Analisys, 30, 12911314, 1993.
[9] J.E. Denis, M. ElAlem, K. Williamson, A
TrustRegion Algorithm for LeastSquares Solutions of
Nonlinear Systems of Equalities and Inequalities,
SIAM Journal on Optimization 9(2), 291315, 1999.
[10] S. Eﬀati, A.R. Nazemi, A new methid for solving a
system of the nonlinear equations, Applied
Mathematics and Computation, 168, 877894, 2005
[11] Goldberg, D.E. Genetic algorithms in search,
optimization and machine learning. Addison Wesley,
Reading, MA, 1989.
[12] W. Gragg, G. Stewart, A stable variant of the secant
method for solving nonlinear equations, SIAM Journal
of Numerical Analisys, 13, 889903, 1976.
[13] J. M. Ortega and W. C. Rheinboldt, Iterative solution
of nonlinear equations in several variables. New York:
Academic Press, 1970
[14] W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P.
Flannery, Numerical Recipes in C: The Art of
Scientiﬁc Computing, Cambridge University Press,
2002.
[15] Steuer, R. E. Multiple Criteria Optimization. Theory,
Computation, and Application. Wiley Series in
Probability and Mathematical Statistics: Applied
Probability and Statistics. New York: John Wiley
Sons, Inc, 1986.