You are on page 1of 4


Kiyotaka Yamamura

Department of Computer Science, Faculty of Engineering,

Gunma University, 1-5-1Tenjin-cho, Kiryu-shi, 376 Japan

Abstract - This paper presents some simple and practical algo- However, these curve tracing algorithms are not widely used in prac-
rithms for tracing implicitly defined solution curves. These algorithms tical applications. One of the reasons is that the theory and the
use hyperspheres instead of hyperplanes that are used in the typical programming of these algorithms seem t o be difficult for practical
predictor-corrector algorithms. Effective techniques for preventing the designers and scientists who are not familiar with the curve tracing
"reversion" phenomenon of the curve tracing are also proposed. The algorithms .
proposed algorithms are geometrically clear and can be easily pro- In this paper, we propose some simple algorithms for tracing the
grammed. solution curves. These algorithms have geometrically clear interpreta-
tions and can be easily programmed. Therefore, they will be widely
used by many scientists. Another advantage of the algorithms is that
I. INTRODUCTION the "reversion" phenomenon of the curve tracing (that often occurs
Let us consider a system of n nonlinear equations in n + 1 variables in the BDF algorithm) is prevented by special techniques. It is also
shown that the algorithms can be easily implemented on the existing
f1(21,2z, .. .,2",2"+1) = 0 circuit simulators such as SPICE.
f ~ ( l l r ~ ~ i . . . r ~ n r + n=
+ l )0
fn(2lr22,...,2n,+n+l) = 0. (1)
The typical predictor-corrector algorithms use tangents as predic-
Since there is one more variable than there are independent equations, tors and hyperplanes as correctors as shown in Fig.1 [1],[2]. In the
the set of all z = ( X ~ , + ~ , ~ ~ . , Z ~ that
, Z , satisfy
, + ~ ) ~(1) will gener- predictor step, the next point is predicted by a tangent to obtain a
ally consist of one or more solution curves in the (n 1)-dimensional good starting approximation of Newton's method. In the corrector
Euclidean space. step, a system of nonlinear equations is solved by Newton's method
To find all solutions of (1) is a basic problem that is widely encoun- to get back onto the solution curve. The stepsize of the predictor (the
tered in science and engineering. For example, to obtain the driving- length of the tangent) is pertinently determined; for example, if New-
point characteristics and transfer characteristics of nonlinear resistive ton's method does not converge or seems t o require many iterations,
circuits, it is necessary to solve an equation of the form ( l ) , where the stepsize is shortened. Also, if the number of Newton iterations is
q,+1 denotes either the driving-point voltage or current. small, the stepsize is expanded. In the BDF algorithm, an efficient
Another common situation that calls for the solution of (1) is the stepsize control algorithm is utilized [12].
application of the homotopy method (or the continuation method) for However, the predictor-corrector algorithms using tangents and hy-
solving a system of n nonlinear equations in n variables perplanes are not simple and their programming is complicated. An-
other shortcoming of these algorithms is that they become extremely
f = 0, (2) slow in the neighbourhood of sharp turning points of the solution
curves. In Fig.2, the stepsize of the predictor is made s-naller and
where f : R" -+ R" is a C2 mapping and z E R". In the homotopy smaller until the hyperplane intersects the solution curve.
method, we introduce an extra variable t n + l 5 t , in addition to z = In this section, we propose a new algorithm that uses hyperspheres
( ~ 1 ~ 2..2.,z,)~,
, and define a new system of n equations instead of hyperplanes. As shown in Fig.3, a sphere that encloses
a part of the solution curve always intersects the solution curve at
h ( z , t )= 0 (3) least two times. Hence, the corrector equation always has at least two
in n 1 variables. The mapping h is called a homotopy. One of the solutions.
most common homotopies is the Newton homotopy Let c E R"+' be a point on the solution curve. Then the equation
describing the (n 1)-dimensional sphere centred at c with radius P
h ( z , t )= f(.) - (1 - t)f(zO), (4) is given by

where zo is any convenient constant vector. At t = 0, z"is a solution

of (3), and at t = 1, (3) is reduced t o (2). Hence, by tracing the
(21 - ci)' + ( ~ -2 cz)' +. .. + (zn+i - Cn+1)' = r'. (5)
solution curve starting from (z", 0), we can obtain the solutions of (2) Hence the intersections of the sphere and the solution curve (A and B
at t = 1. This homotopy method is known to be globally convergent for
many practical problems including DC analysis of nonlinear circuits.
Many algorithms have been presented for tracing the solution curves
[1]-[17]. These curve tracing algorithms are roughly divided into two
types; one is the predictor-corrector algorithms and the other is the
piecewise-linear algorithms. These algorithms are very efficient a t least
for small problems. Especially, the backward-differentiation formula
curve tracing algorithm (in this paper, we call this algorithm the BDF
algorithm for short) proposed by Ushida and Chua 1121 is one of the
most effective and reliable algorithms for tracing solution curves that
have sharp turning points. Fig. 1 Fig. 2 Fig. 3

Fig. 4 -
in Fig.4) are the solutions of the following system of n
Fig. 5

+ 1 equations
(SI -
Fig. 6

equation in the spherical algorithm is

f(z)- (1 - t ) f ( z O )= 0
Fig. 7

+ ( 2 2 - C 2 l 2 + + (In- c,)2 + (t - c+,l 12 - 7.2, (7)


fi(z19z2,. ’ 7 I n ,z n + l ) = 0
. f 2 ( ~ 1 , ~ 2 9 . .. zn,zn+1) = 0 Let the the Jacobian matrix of f be J . Then the ( n + 1) x ( n + 1)
Jacobian matrix of (7) is obtained by adding one row and one column
to the n x n matrix J .
In the existing circuit simulators such as SPICE, programs of New-
ton’s method for f(z) = 0 are already described. Therefore, the
spherical algorithm is easily programmed through a little modifica-
As shown in Fig.5, the solution curve can be traced by successively
tion of the existing programs. Thus, the spherical algorithm can be
solving the systems of nonlinear equations ( G ) . In each step, the s-
readily implemented on the circuit simulators. 0
lution that has just been obtained is used as the centre of the new
sphere. The radius of the sphere is shortened if Newton’s method
does not seem to converge in the corrector step. The initial radius in 111. THE IMPROVED SPHERICAL ALGORITHMS
each step is determined by the stepsize control algorithm proposed in
P21. Let us consider Fig6 again. If the sphere is sufficiently small, New-
Each corrector equation has at least two solution, one of which lies ton’s method will converge to the genuine solution B. However, using
in the forward direction and another of which lies in the backward small spheres is time consuming, and it is desirable to use large spheres
direction. In order to make the Newton iteration converge to the to make the algorithm efficient. In such a case, Newton’s method may
desired solution in the forward direction (that is called the genuine not converge to the genuine solution B. Instead, it may converge to
solution in this paper), the predictor algorithm is executed. There the solution A (that is called the spurious solution in this paper) that
are many predictor algorithms, the most efficient one of which is the lies in the backward direction as shown in Fig.7. Such a phenomenon
BDF predictor algorithm proposed by Ushida and Chua [12]. In order is called the “reversion” of the curve tracing.
to make the algorithm simple, we consider to use the first (or second) If the radius of the new sphere is the same as the previous one, this
order BDF predictor algorithm. Then, as shown in Fig.G, the predicted reversion can be detected because the spurious solution coincides with
value is obtained by extrapolating the two (or three) previous solutions the precedingly obtained solution. However, if the radius changes, the
that are on the solution curve. reversion cannot be detected because the spurious solution does not
This is the basic form of the spherical algorithm proposed in this coincide with the preceding solution as shown in Fig.8. In this case,
paper. This algorithm is simple and geometrically clear, so it may be the curve tracing reverses from this point. In the BDF algorithm,
widely used by practical designers. this reversion phenomenon often occurs in practical executions. In
this section, we propose two improved algorithms that overcome this
Remark 1: The spherical algorithm proposed in this section is not In the first improved algorithm, we move the centre of the sphere
a new algorithm because it is essentially equivalent to the first order a little so that the new sphere passes the preceding solution point.
BDF algorithm. Mr. Yasuaki Inoue of Sanyo Electric Co., Ltd. was In Fig.9, point A is the precedingly obtained solution, point D is the
the first to point this out [17]. However, the concept “sphere” makes solution that has just been obtained, point C is the centre of the new
the algorithm geometrically very clear and gives considerable insights sphere, and point B is the next solution that we are going to seek.
to the algorithm. As shown in t,he next section, this geometrical inter- The centre C is determined by the first or second order interpolation
pretation makes it easy to detect the potential problems of the BDF or extrapolation.
algorithm and to find the techniques for overcoming them. 0 Fig.10 describes how to reduce the radius of the sphere when New-
ton’s method does not converge in the corrector step. For simplicity,
we will halve the stepsize of the predictor if convergence does not oc-
Remark 2: The advantages of the spherical algorithm are its sim-
cur. Suppose that we first use a sphere that is the same size as the
plicity and ease of programming. In circuit simulation, there is another
preceding one and Newton’s method does not converge to the genuine
solution B. In order to halve the stepsize of the predictor, we use a
Existing circuit simulators such as SPICE use Newton’s method or
sphere with radius 3/4 times as large as the preceding one. If conver-
its variants to solve nonlinear equations. However, Newton’s method
gence does not occur again (for example, Newton’s method converges
is not globally convergent and requires a good initial guess for conver-
to the spurious solution A), we reduce the radius 5/G times to make the
gence. Guessing a solution is often a difficult task, and many circuit
designers experience difficulties in finding DC solutions using the cir-
cuit simulators. The homotopy methods are proven to be globally
convergent for nonlinear circuit equations [9],[13],[15],hut they have
not been widely used because the present circuit simulation depends
largely on the existing circuit simulators. Hence, it is desirable to
improve the convergence through a little modification of the existing
programs in the circuit simulators.
Let (2) be a circuit equation and consider to solve (2) by the ho-
motopy method using the Newton homotopy (4). Then the corrector Fig. 8 Fig. 9

mw ”
,... ....,,
A ......._._._..
. . ....

.... ,.,,_.....

Fig. 10

Fig. 11
example, the direction of the variable that exhibits the maximum vari-
ation in the predictor step) is often more effective than the spherical
algorithm. The cylindric algorithm is an intermediate concept be-
tween the standard continuation method and the spherical algorithm,
and often more efficinet than the spherical algorithm, although not
It is also possible to prevent the reversion phenomenon in the el-
IiDtic algorithm and the cylindric algorithm. Details are discussed in


%J In this paper, we have proposed some simple algorithms for tracing

solution curves. The proposed algorithms are geometrically clear and
Fig. 12
can be easily programmed, so they can be widely utilized by many
scientists. Also, the reversion of the curve tracing is prevented in the
improved algorithms.
We applied the homotopy method using the spherical algorithms to
various types of nonlinear resistive circuits. When we applied the basic
spherical algorithm proposed in Section I1 (that is equivalent to the
first order BDF algorithm), the reversion phenomenon often occured.
In these cases, the curve tracing went back t o t = 0 and solutions could
not be obtained. However, when we applied the improved spherical
algorithms, the reversion occured but it was always detected, and solu-
Fig. 13 Fig. 14 tions could be obtained. When we used small spheres, the algorithms
found the solution steadily but not rapidly. When we used very large

spheres, the algorithm found the solution very rapidly when it worked
well. However, sometimes the phenomenon described in Fig.14 oc-
cured and solutions could not be obtained.
The computational efficiency of the algorithms depended largely on
...... the parameters such as the starting point, the initial stepsize, the max-
imum stepsize, and the maximum number of Newton iterations that
is allowed. The answer to the questions “Which improved algorithm
Fig. 15 Fig. 16 is more efficient?” and “What is the most efficient way to determine
the parameters?“ is “Case by case.” Anyway, since the reversion phe-
nomenon is prevented, we can use large spheres and take a chance.
stepsize 1/2. Repeating this process, the predicted point approaches
the genuine solution, and convergence will occur in a finite number of
steps. Notice that all spheres pass the point A, so that the reversion ACKNOWLEDGMENT
can be detected. The author would like to thank Mr. Yasuaki Inoue of Sanyo Elec-
Next, we shall propose the second improved algorithm. In this al- tric Co., Ltd., Gunma, Japan, and Prof. Akio Ushida of Tokushima
gorithm, we use a sphere which passes the solution point that has just University, Tokushima, Japan, for their fruitful discussions and com-
been obtained. Fig.11 illustrates this algorithm. In this algorithm, ments. The author also acknowledges Prof. Kazuo Horiuchi and Prof.
the centre of the sphere is always determined by (the first or second Shin’ichi Oishi of Waseda University, Tokyo, Japan, for their rnnstant
order) extrapolation. Since the spurious solution is known, we can encouragement.
also detect the reversion in this case.
In the spherical algorithm proposed in Section 11, the radius of the
sphere determines how far the algorithm moves in one step. In the REFERENCES
second improved algorithm, the diameter determines the stepsize as
shown in Fig.12. [l] C.B.Garcia and W.I.Zangwill, Pathways to Solutions, Fixed
Now we shall compare the first and the second improved algorithms. Points, and Equilibria, Englewood Cliffs, NJ: Prentice-Hall, 1981.
As shown in Fig.12, the second algorithm uses smaller spheres if the [2] E.Allgower and K.Georg, “Simplicial and continuation methods
stepsize of the predictor is the same. In all of the predictor-corrector for approximating fixed points and solutions to systems of equa-
algorithms, the following potential dangers can occur; the algorithm tions,” SIAM Review, v01.22, no.1, pp.22-84, Jan. 1980.
goes to a different solution curve; it cycles; it attaches itself to a [3] D.F.Davidenko, “On the approximate solution of a system of non-
loop (Fig.13). As shown in Fig.14, the second algorithm uses smaller linear equations,” Ukraine Math. Zurnal, vo1.5, pp.196-206, 1953.
spheres, and this reduces the possibility that those dangers occur. [4] YShinohara, “A geometric method of numerical solution of non-
However, in the second algorithm, the centre of the sphere is always linear equation and error estimation by Urabe’s proposition,”
determined by extrapolation. As shown in Fig.15, the centre may be RIMS, Kyoto University, vo1.5, pp.1-9, 1969.
far away from the solution curve in the vicinity of sharp turning points. [5] F.H.Branin, “Widely convergent method for finding multiple solu-
Therefore, the second algorithm will require a good predictor. tions of simultaneous nonlinear equations,” IBM J. Res. Develop.,
vo1.16, pp.504-522, 1972.
[6] K.S.Chao, D.K.Liu and C.T.Pan, “A systematic search method
IV. THE ELLIPTIC ALGORITHM AND THE CYLINDRIC for obtaining multiple solutions of simultaneous nonlinear equa-
ALGORITHM tions“, IEEE R u n s . Circuits Sysi., vol.CAS-22, no.9, pp.748-753,
As the variants of the spherical algorithm, we can also consider al- Sept. 1975.
[7] E.Ikeno and A.Ushida, “The arc-length method for the com-
gorithms that use ellipses or cylinders. Fig.16 illustrates the elliptic
putation of characteristic curves,” IEEE Duns. Circuits Syst.,
algorithm. The use of ellipses that are long in suitable directions (for
vol.CAS-23, no.3, pp.181-183, March 1976.

[8] L.O.Chua and A.Ushida, “A switching-parameter algorithm for
finding multiple solutions of nonlinear resistive networks,” Int. J.
Cir. Theor. Appl., ~01.4,no.3, pp.215-237, July 1976.
[9] T.Ohtsuki, T.Fujisawa, and S.Kumagai, “Existence theorems and
a solution algorithm for piecewise-linear resistor networks,” SIAM
J. Math. Anal., vo1.8, no.1, pp.69-99, Feb. 1977.
[lo] M.J.Chien and E.S.Kuh, “Solving nonlinear resistive networks
using piecewise-linear analysis and simplicial subdivision,” IEEE
Trans. Circuits Syst., vol.CAS-24, no.6, pp.305-317, June 1977.
[ll] K.S.Chao and R.Saeks, “Continuation methods in circuit analy-
sis,” Proc. IEEE, vo1.65, pp.1187-1194, Aug. 1977.
[12] A.Ushida and L.O.Chua, “Tracing solution curves of non-linear
equations with sharp turning points,” Int. J. Cir. Theor. Appl., In general, each equation in (Al) is nonlinear in only a few variables,
v01.12, no.1, pp.1-21, Jan. 1984. and is linear in other variables. This is especially the case when the
[13] K.Yamamura and K.Horiuchi, “A globally and quadratically con- circuit equations are written in the hybrid n-port representation form.
vergent algorithm for solving nonlinear resistive networks,” IEEE For example, consider a circuit containing n/2 bipolar transistors ( n
Tmns.Comput.-Aided Des. Integrated Cirsuits Syst., vo1.9, no.5, is assumed to be an even number), linear resistors and independent
pp.487-499, May 1990. sources. If this circuit is described by the hybrid n-port represeuta-
[14] L.Vandenberghe and J.Vandewalle, “Variable dimension algo- tion, each function f j ( q , x ~’ ,. . , zn) is nonlinear in only two variables.
rithm for solving resistive circuits,” Int. J . Cir, Theor. Appl., That is, f1 and fz are nonlinear in 21 and X Z , fs and f4 are nonlinear
vo1.18, no.5, pp.443-474, Sept. 1990. in t3 and q ,and so on.
[15] L.TrajkoviC, R.C.Melville, and S.C.Fang, “Passivity and no-gain Let us consider the piecewise-linear approximation of f1 that is non-
properties establish global convergence of a homotopy method for linear in q and 2 2 only. In this case, we do not have to divide the
DC operating points,” in Proc. IEEE Int. Symp. on Circuits and n-dimensional operating region into 10“ linear regions. Instead, we
Systems, New Orleans, LA, pp.914-917, May 1990. divide the region into lo2 linear regions as shown in Fig.Al. This fact
[16] L.TrajkoviC, R.C.Melville, and S.C.Fang, “Finding DC operating can be easily verified.
points of transistor circuits using homotopy methods,” in Proc. The new sign test proposed in this appendix is a test applied to the
IEEE Int. Symp. on Circuits and Systems, Singapore, pp.758-761, long rectangles in Fig.Al. We first apply the sign test to f1 on 10’
June 1991. long rectangles and eliminate regions that do not contain a solution of
[17] Y.Inoue, “DC analysis of non-linear circuits using solution tracing Fl(z) = 0. It is easily seen that this test provide a necessary condition
circuits,” in Proc. European Solid State Circuit Conf., Milano, on the existence of a solution in each long rectangle. After the test to
pp.41-44, Sept. 1991. f l , we apply the similar test to fz, f3,. . . ,f n , successively. In this case,
[18] K.Yamamura, “Simple algorithms for tracing solution curves,” the total number of performance of the sign test is 10’n. Therefore,
IEICE Technical Report, NLP91-51, pp.47-53, Nov. 1991. the total numbers of additions and comparisons in the worst case are
proportional to n3.
After this first test, we perform the second test. We perform the sign
APPENDIX test proposed in [All to the linear regions (small rectangles) that were
not eliminated by the first test. Then linear simultaneous equations
Since there is some space, the author would like to introduce his re- are solved on the survived regions.
cent result on finding all solutions of piecewise-linear resistive circuits We applied this new algorithm to a multi-diode circuit that is de-
briefly. This work is a developed version of the preceding paper [All. scribed by a system of 10 equations. The number of linear regions is
Details are discussed in [A2]. 10”. We first applied the Yamamura and Ochiai algorithm. Then the
Let us consider a nonlinear resistive circuit described by a system total computation time was 526,393 seconds (about six days). Next we
of n nonlinear equations applied the new algorithm. When we applied the first test, only 6,046
regions passed the test. Then we applied the second test to these re-
fI(Zl,Q,...,%) = 0 gions, and 4,865 regions passed. By solving linear simultameous equa-
fz(zl,zz,...,+n) = 0 tions on these survived regions, we could obtain the solutions. The
total computation time was only 14 seconds. That is, the computation
f”(tl,2.2,“‘,%) = 0. (All time was reduced
Assume that this system of equations has a separable mapping. In I from 6 days to 14 seconds. I
the piecewise-linear analysis, (Al) is approximated by a continuous
piecewise-linear equation F ( z ) = 0. For simplicity, we assume that We have applied the proposed algorithm to many circuits (transistor
the number of linear regions in R” is 10”. circuits, multi-tunnel diode circuits, neural networks, and fixed point
In [All, Yamamura and Ochiai propose an efficient algorithm for problems in [2],[12]). In all cases, similar improvement of computa-
finding all solutions of F ( z ) = 0. In this algorithm, a simple sign test tional efficiency could be achieved.
is performed for eliminating many linear regions that do not contain Since this algorithm also provides an estimation of the total com-
a solution. Therefore, we can find all solutions without solving linear putation time at the early stage of the execution, it can be applied to
simultaneous equations on every linear region. large scale problems without anxiety.
However, although the sign test itself is very simple and requires at
most O ( n 2 )additions and comparisons per one region, it is performed
10” times. This makes the total computation time very large when n
is large. [All K.Yamamura and M.Ochiai, “An efficient algorithm for finding
In this appendix, we propose a new sign test whose total compu- all solutions of piecewise-linear resistive circuits,” IEEE Trans.
tational complexity is proportional to n3 (not 10”nZ),and introduce Circuits Syst., vo1.39, March 1992; also in Proc. IEEE Int. Symp.
this new test to the Yamamura and Ochiai algorithm. That is, in the on Circuits and Systems, Singapore, pp.3039-3042, June 1991.
new algorithm, we first apply the new test and eliminate many linear [A21 K.Yamamura, “Finding all solutions of piecewise-linear resistive
regions with a little computational effort. Then we apply the the sign circuits using simple sign tests,” IEICE Technical Report, NLP91-
test proposed in [All to the passed regions. This algorithm is much 85, Jan. 1992.
more efficient than the Yamamura and Ochiai algorithm.