Steward 1962

SIAM Rvxw
Vol. 4, No 4, October, 1962

Printed in U.S.A.
Downloaded 11/28/14 to 129.120.242.61. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
ON AN APPROACH TO TECHNIQUES FOR THE ANALYSIS OF THE

STRUCTURE OF LARGE SYSTEMS OF EQUATIONS
DONALD V. STEWARD
1. INTRODUCTION
THE MAIN CONCERN OF THIS PAPER i$ with very large systems of algebraic
equations (linear or nonlinear) in which the number of variables appearing in
any equation is small compared to the total number of variables in the system.
Such problems frequently arise in engineering as the numerical solution of a
large set of simultaneous physical and economic constraints. Compared with
the first algorithm likely to be conceived for the solution of such a system, it is
frequently possible to make many orders of magnitude reduction in required
computer time by further analysis. Such reductions in computer time would
make tractable many problems presently considered intractable. However, the
cost of this analysis itself may be prohibitive. Indeed, the analysis of very large
systems is a sizable data handling task. The work reported has been motivated
by the desire to establish formal techniques suited to computer assistance in
the analysis. This paper, in itself, certainly does not satisfy this goal. It is hoped,
however, that the work does constitute a first step which will encourage further
investigation of the problem.
There exist several approaches to the analysis of systems of equations through
the use of some "topology." Harary [1] derives his topology from graphs of the
permutation group describing the determinant. Mason [2] derives his topology
from analogy to electrical networks. Kron [3] apparently uses his insight into
the physical problem to understand the structure of the equations. Tribus [4]
uses diagrams, but does not offer a formalism. Simon [5] discusses partitioning
associated with causal relations. The method of this paper was derived inde-
pendently [6], and seems to the author to be more suited to the purposes of
the paper than any of the aforementioned approaches.
2. MOTIVATION Of CONCEPTS
To introduce the concept of information flow, two examples, (1) and (2),
of the solution of a set of equations are considered. On the left are the functional
representations of the equations. On the right are the same equations in the
form of an array indicating those variables appearing in each equation.
a c
Fl (x,,) 0 1 X
(1) F.(x x) 0 2
F3(xb, xc) 0 3 X
This work was essentially completed while the author was employed by General Elec-
tric Company, Atomic Power Equipment Department, San Jose, California. Received by
the editors, August 17, 1961 and in revised form July 27, 1962.
Burroughs Corporation, Burroughs Laboratories, Paoli, Pennsylvania.
321
322 D, V. STEWARD
a b c
F (xo, x) 0 1
(2) F. (x, x) 0 2
F3(xc xa) 0 3 X
The system of equations of (1) would be solved as follows. Independently of
any other equations, equation 1 can be solved for the number represented by
xa. Once a number for x is available, equation 2 may be solved for the number
represented by xb. With a number for xb available, equation 3 may be solved
for the number represented by xc. Let Gla be used to represent the solution of
equation 1 for the variable xa, etc. Then, the above sequence of operations may
be represented in functional form as follows"
number
(3) a (xo) x
number
Go() xo
A sequence of substitutions and solutions is said to imply a flow of information.
This flow of information may be described as follows" an equation symbol fol-
lowed by a variable symbol implies the solution of that equation for that var-
iable; a variable symbol followed by an equation symbol implies the substitution
of that variable in that equation. The flow of information is then described by
a sequence of alternating equation symbols and variable symbols called a path.
A path beginning and ending with an equation symbol will be called a chain. For
example, the chain l a263 implies that equation I is solved for xa, which is substi-
tuted in equation 2, which is solved for xa, which is substituted in equation 3.
Thus,
a b c
1 X
(4) 2 X-+ X chain la2b3
3 X-’-> X
A vertical line through a variable to an equation represents a substitution (a
variable symbol followed by an equation symbol). A horizontal line through an
equation to a variable represents a solution of that equation for that variable
(a variable symbol followed by an equation symbol).
For the system of equations of (2), there is no equation which can be solved
independently of the others. However, the solution may proceed either by the
process of iteration or by the process of elimination. Both processes will be shown
to imply the same flow of information.
ANALYSIS OF THE STRUCTURE OF LARGE SYSTEMS OF EQUATIONS 323
For either iteration or elimination, the first consideration is for which variable
each equation is to be solved. This variable is called the output variable (or
dependent variable) of that equation. The remaining variables of the equation
are called input variables (or independent variables) of the equation. The list
giving the output variable for each equation is called the output set. For example,
it may be appropriate to solve equation 1 for xb, equation 2 for xc, and equation
3 for xa, which would be described by the output set (lb, 2c, 3a).
An output set must be such that:
1. Each equation has exactly one output variable.
2. Each variable appears as the output variable of exactly one equation.
An equation may contain some variables for which it cannot be solved
uniquely, and there may exist a number of possible output sets meeting the
above requirements. Any output set for which each of the equations can be
solved for the required output variable is called an admissible output set. If a
system of equations is to be solved, then there must be at least one admissible
output set. Later, the manner of proceeding from one output set and its proper-
ties to any other output set and its properties will be described. Thus, for much
of the work, any output set will suffice, provided there exists an admissible output
set.
In (2), either (i, 2c, 3) or (le, 2, 3c) constitutes a valid output set. To
proceed with the illustration, (I, 2c, 3) is arbitrarily chosen. This output
set implies that
()
(5) JG.o(x) xo
G(xo) xo.
Let this output set be used to solve the system of equations of (2) by iteration.
The iteration begins with an arbitrary guess at a number for one of the variables,
say xa 0. Using this number for xa, equation 1 can be solved for a number
represented by xb, (lb). This number is substituted for x into equation 2, (b2),
to solve for a number represented by xc, (2c), which is substituted into equation
3, (c3), to solve for a new number represented by Xa, (3a), which is substituted
back into equation 1, (al). (It will not be considered here whether the iteration
will converge.) This procedure, giving the chain lb2c3al, may be represented in
functional form as
G (x()) x ()
number
(6) O,o (x (’)) xo(’)
number
number
i + 1 --+ i G. (X(i)) Xa
(iT1)
324 D.V. STEWARD
Let the same output set, (lb, 2c, 3a), be used to solve the same equations by
elimination. Equation 1 is solved for xb as an expression in x, (lb), which is,
in turn, substituted for xb into equation 2, (b2), to solve for xc, (2c), as an
expression in x,. Thus, x has been eliminated from equations 1 and 2. Now
this expression may be substituted for xc into equation 3, (c3), which eliminates xo,
leaving an equation for x,, (3a), as a function of xa. This equation may be solved
for the exact number represented by Xa. This number is then substituted for xa
into equation 1, (al), to solve for the number represented by x, which is, in
turn, substituted into equation 2 to solve for the number represented by xo.
The chain describing this process is, again, lb2c3al. This process may be repre-
sented in the following functional form:
G() x
symbol number
(7)
Gc(x) G.o(G(x,)) G:(x,) x
symbol
, (.) G (G (.)) (x.)
Note that, for both iteration and elimination, given the same output set (lb,
2c, 3a), the me chain, lb2c3al, representing the same sequence of substitu-
tions, is obtained. Iteration is performed by the substitution of numbers;
elination is performed by the substitution of symbolic expressions.
The chain lb2c3al, obtained above, may be diagrammed as follows:
a b c
1
(8) 2 XX chain (loop) lb2c3al
3 X X
Note that this chain closes back upon itself; that is, the equations represented
by the first and last elements of the chain are the same. Such a chain is called a
loop. The presence of a loop implies that no equation appearing in the loop may
be solved independently of the other equations in the loop. Unknown informa-
tion must be carried around the loop before the equations in the loop can be
solved. For iteration, the unsown information is carried as the error in a num-
ber. For elimination, the unknown information is carried symbolically.
Solving the equations in the loop may begin with any equation appearing in
the loop. In (8), the procedure may have begun with a solution of equation 2
for x, then of equation 3 for x, etc. This sequence would give the loop 2c3alb2.
Equivalent forms of the same loop can be said to exist if, when the last elements
are dropped, the forms contain the same elements in the same cyclic order.
Given any loop, all of its equivalent forms may be quickly generated. Thus,
only one form of a loop need be known. The form of the loop beginning and
ending with the lowest numbered equation is referred to as the canonical form
of the loop. For example, lb2c3al, 2c3alb2, and 3alb2c3 are equivalent forms of
the loop whose canonical form is lb2c3al.
Given different output sets, different loops will be obtained. However, in a
following section, relationships between loops and output sets will be shown,
such that, given any output set and its loops, any other output set and its loops
may be directly generated.
3. A FIRST OUTPUT SET
The method of obtaining a first output set is considered in this section. The
method of obtaining the loops, given an output set, is considered in the next
section. In a following section, relationships between loops and output sets
are shown, such that, given one output set and its loops, any other output set
and its loops may be directly generated.
The following theorem is proved: Given a system of equations containing as
many variables as equations, if there does not exist an output set, then there exists
a subset of the equations containing fewer variables than equations.
Some observations are made first. Given that the system of equations has a
subset of equations containing fewer variables than equations, usually, but not
always, there does not exist a unique solution. For, usually, at least one equation
of the subset is not independent, or else the variables are overspecified. If the
variables are not overspecified, then, at best, all of these variables might be
eliminated from the remaining equations. Then, the remaining equations would
contain more variables than equations. Such a system of equations cannot usually
be solved uniquely. However, consider the exception:
(9) 1 (x, y, z real)

2z 2.
Thus, if no output set exists, further analysis of the equations themselves is
required.
The proof of the above theorem will be in the form of a program flow chart
(Fig. 1) which will lead either to an output set or to the conclusion that there
exists a subset of equations containing fewer variables than equations.
The proof begins with the array showing the variables appearing in each equa-
tion. The rows correspond to equations, the columns to variables. The ’s
indicate which variables appear in each equation. is placed beside an X to
indicate that the X represents the output variable of an equation. Check marks
(w/) will be placed at the heads of rows and columns during the program. The
rules for an output set require that exactly one be in each row and exactly
326
D.V. STEWARD
one be in each column. There may exist many output sets satisfying these
conditions. Only one need be found.
It must now be shown that this process cannot fall into an infinite loop. Each
time the process passes through step 2, a new equation is encountered. Each
time the process passes through step 1, a new output variable is assigned. There
is a finite number of variables and a finite number of equations. Thus, infinite
loops are impossible.
For example, assume that, at some time, the process arrives at step 1 with
the following condition"
a b c d e f g
1 X X
2 Xf X
3 X X X
(10) 4 X
5 X X,,,::,
v’6 X X,,,::,
7 X X X
Starting with step 1, the following condition is reached"
a b c d e f g
/1
2 / T x
X--X---X
(11) /4 Xf
5 X X@
6 X X,,,,:,
7 X,f, X X
This condition gives the output set (le, 2b, 3g, 4a, 5d, 6f, 7c). Note that the
output has been moved along the path g3ela4, from an unassigned variable to
an unassigned equation. Note also that this is a special case of the assignment
problem.
4. GENERATION OF LOOPS
The method of generating loops, given a system of equations and an output
set, will be demonstrated.
Consider, as a new example, the following three equations with the output
set (la, 2b, 3c)
a b c
Fl (x,, xc) 0 1 X X
(12) F2(xa,x,,xc)-0 2 X Xe X
F3(xb, x) 0 3 X Xe
328 D. . STEWARD
Let i represent the output variable of equation i. Then, ai is defined to be

ij, if variable appears in equation j and i j.
a.
[0, otherwise.
The expression i j is called a link. Let a. become the ij element of a matrix.
This matrix is called the lint matrix. For example, consider the element
(that is, the element in row 1, column 2) of the link matrix (13) for the system
of equations and output set of (12). Then, i 1. In the output set (la, 2b,
3c), 1 a. Further, xa appears in equation j 2. Thus, a2 ij la2.
Thus, for (12), the following link matrix is obtained:
1 2 3
(13) 2
(With reference to graph theory, it can be seen that, if the non-zero elements
are set to Boolean l’s, (13) represents an adjacency matrix. In a following
section on "Partitioning," the Boolean form will be used. However, elsewhere
the interest will be not simply in the existence of chains, but in the tracing of
chains. The letters representing the variables need not be included, since the
output set is known. However, the letters are included here for clarity.)
A chain from i to j and a chain from j to / imply the existence of a chain
from i to k. In this way, chains describing sequences of substitutions may be
built. This leads to the following definition of the multiplication of chains"
(14) (i, j). (j. k) iq, jg..., k;
that is, if the last element of the left chain matches the first element of the right
chain, the first element of the right chain is dropped, and the chains are joined
to form the product. If the last element of the left chain does not match the
first element of the right chain, the product does not exist. For example
1a2.263 1a263
(15) 1a2b3 3c I l a2b3c 1
lla2.3cl (does not exist).
The analysis will not, at this time, deal with chains which contain subchains
which form a loop. For example, the chain l a263c2b4 has the subchain 263c2,
which is a loop. A chain containing no subchain which is a loop is referred to as
a proper chain. To the rules above, defining multiplication of chains, is added a
rule that states" If the product formed according to the rules above would form
an improper chain, then that product is said not to exist. For example,
la2b3.3c2b
" (does not exist, since la2b3c2b4 would contain 2b3c2 as a subchain).
The number of distinct variables represented by elements in a chain is called
the order of the chain. As has been shown, a first-order chain is a linlc.
It must be allowed that an element of the matrix may be a set of chains, as
there may be several chains which extend from i to j. The addition of two such
elements will be implied to mean the union of these sets. Multiplication will be
implied to mean the Cartesian product of sets using the above product of chains
(that is, the set of products of all combinations of one chain taken from the left
set times one chain taken from the right set).
The set of all second-order chains from i to j is the set of chains iikj, for
all ]. The definitions of addition and multiplication of elements cited above are
applied to the usual definition of multiplication of matrices to define the multi-
plication of link matrices. Then, the matrix containing all second-order chains
is the square of the link matrix. Similarly, all nth order chains occur in the nth
power of the link matrix. If there are n equations, powers of the link matrix
greater than n are empty, as no proper chain can intersect more than n equa-
tions. Loops are chains from i to i, and thus appear on the diagonal. Thus,
0 la2 0
[a-] 0 0 2b3
3cl 3c2 0
(16) [a.] 2b3cl 2b3c2 0

0 3cla2 3c263
la2b3cl 0 0
[ai.] 0 2b3cla2 0
0 0 3cla2b3
Note that, in the mth power of the matrix, each loop appears in each of its m
equivalent forms. For example, the loops 263c2 and 3c263 appearing in the second
power are equivalent, and the loops la2b3cl, 2b3cla2, and 3cla2b3 appearing
in the third power are equivalent. Carrying any chains which will not contribute
to the canonical form of a loop can be avoided by adding another rule to those
defining multiplication: If the product of two chains as defined previously would
give a chain such that an equation appears with lower number than the number
of the first equation in the chain, it can be said that this product does not exist.
:For example, 5b8c6.6d3e7 ;2 (does not exist, as 5b8c6d3e7 would contain a
3, and 3 < 5). This added rule can significantly reduce the number of operations
required to obtain the mth power.
330 D.V. STEWARD
In the example above, there are two distinct loops which, stated in canonical
form, are 263c2 and la2b3cl. These loops can be displayed as follows:
a b c a b c
1 Xq X 1 X( X
la2b3ci
(17) 2 2 X--X X
3 X--- Xq
5, LOOPS AND OUTPUT SETS
The foregoing has demonstrated the method of obtaining an output set, and
the method, given the output set, of obtaining the loops associated with that
output set. Relationships between loops and output sets are considered now,
such that, given any output set and its loops, any other output set and its loops
may be directly generated.
A system of equations involving only one loop is considered first"
a b c
1 X--*X
output set (lb, 2c, 3a)
(18) 2 X--- Xq loop lb2c3al
3 X X
Let the output of equation 1 be moved from b to a. Now, both equation 1 and
equation 3 have x as an output. Thus, the output of equation 3 must be moved
from a to c. This leaves both equation 3 and equation 2 with the same output
variable, xc. Thus, the output of equation 2 must be moved from c to b. But
b was vacated by the first move in equation 1. Thus, the procedure is complete,
and the new output set can be displayed as follows:
a b c
1 X+-- X
T output set (la, 2b, 3c)
(19) 2 X’- X loop la3c2bl
3 X
Note that the output relabelling has been carried backwards around the loop.
The new loop, 1a3c2b1, is the reverse of the loop around which the relabelling
was carried. The new output set, (la, 2b, 3c), is obtained by replacing the out-
put variable of each equation that appears in the new loop by the variable ap-
pearing immediately after that equation in the loop.
It is clear that any transformation from one output set to another must carry
the relabelling of outputs around one or more loops.
There may be more than one loop. Some manner of transforming each of the
loops by relabelling the output around one of the loops, called the relabelling
loop, must be established. First, consider the following example:
a b c
1 X, X
output set la, 2b, 3c)

(20) 2 X >X,< X loops la2b3cl, 2b3c2
3cl
3
The outputs around the loop 2b3c2 will be relabelled. Again, the loop around
which the output is relabelled, 2b3c2, is transformed into its reverse, as follows"
a b c
X ’< X
la21 >X
I.
X?
original loop la2b3cl
reverse of
2b31
X
3b2 >i
2c3 3cl relabelling loop 2b3c2
Common pieces of the paths of the two loops now cancel, and what had been
the loop la2b3cl must now follow along the opposite side of the relabelling loop
2b3c2. The new loop is now:
a b c
1 X, X
(22)
2
la2
X
1 X
]2cl
X loop la2cl
X, X
Thus, the transformation of loops is as follows"
2b3c2 2c3b2 (relabelling loop transforms into its reverse)
(23)
1a2b3c1 --> la2cl
332
D. V, STEWARD
Consider now a formal procedure for the transformation of loops caused by

relabelling the outputs around a loop. The relabelling loop becomes transformed
into its reverse. Any loop containing no equations in common with the re-
labelling loop will not be changed by the relabelling. The transformation of any
loop other than the relabelling loop which contains variables in common with
the relabelling loop is given by the program flow chart of Fig. 2.
In the program of Fig. 2, the loops will be written in the form of the separate
links whose product is the loop. For example, the loop 1a263c1 would be written
as la2, 263, 3cl.
Following through the steps of the chart, it can be seen that these operations
will always lead to loop (s) composed of chains from the new output set.
Using (20), the following results:
[--la2--2b3 3cl ---] loop to be transformed

(24) 3b2--->2c3 reverse of relabelling loop 2b3c2
la2 2cl yielding loop la2cl
6. PARTITIONING
It would be beneficial to partition the solution of large systems of equations

into a partially ordered sequence of solutions of smaller systems of equations,
when this is possible. The following example illustrates this goal"
F (x. x, x) 0
F (x ) 0
Fa(x, x], x) 0
(25) F4(x,,) 0
F (x, x) 0
F (x, x) 0
’ (xo, x., x) 0
These equations can be partitioned into the following partially ordered sequence
of solutions of smaller systems of equations"
334 D.V. STEWARD
Xa X,c Xe Xg
F(x.) o Xa Fl (Xa xc Xe) 0
F(x, x, x) 0
’(xo, xo, ) 0
(26)
Xb Xf
Xd
/, (x,, x) 0
The work may begin either by solving equation 4 for xa, or by solving equations
2 and 6 simultaneously for variables xb and x]. With xb obtained from the so-
lution of equations 2 and 6, equation 5 may be solved for xd. With xa obtained
from the solution of equation 4, and x from the solution of equations 2 and 6,
equations 1, 3, and 7 may be solved simultaneously for x, x,, xg.
Note that this is a partial ordering or precedence relation. The simultaneous
solution of equations 2 and 6 could proceed at the same time as the solution of
equation 4; or, the simultaneous solution of equations 1, 3, and 7 could proceed
at the same time as the solution of equation 5. Given much larger systems of
equations than the trivial example presented, and a computer with the ability
to perform many arithmetic operations simultaneously, it would be very de-
sirable to have a constructive technique for recognizing partitions when they
exist. The following is an application of the method of this paper to this problem.
To partition, those equations having loops in common are grouped together.
Clearly, there will then be no loops in the information flow between these groups
of equations. Therefore, the solutions of these groups of equations obey a prece-
dence relationship.
To partition, those equations having loops in common must be determined,
but the loops need not be identified, nor must improper loops be excluded. Thus,
each link in. the link matrix can be replaced by a 1 to obtain a Boolean (ad-
jacency) matrix. Then, l’s are added to the diagonal so that the sum will be
computed powers of the matrix. Matrix multiplication is now performed with
Boolean arithmetic (that is, 0.0 0.1 1.0 0, 1.1 1, 0 0 0, 0 1 +
1 0 1 1 1). Squaring of this matrix continues, obtaining sums through
the 2nd, 4th, 8th, etc., powers of the original matrix, either until the matrix does
not change, or until the power of the matrix equals or exceeds the order of the
matrix. The resulting matrix has a 1 in the ij element if and only if there is a
path, of any length, from equation i to equation j. Equation k, for example, is
in a common loop with if and only if ak ak 1; that is, there exists a path
from to k and a path from k to 1. The procedure thus begins with/c 1 and
determines all those equations which are in a common loop with /. These
equations are then eliminated from further consideration, and k is advanced to
the next higher equation which has not already been placed in some common
loop. This process is continued until all equations are exhausted. The operation
is readily performed on a computer with Boolean commands.
This grouping of equations is independent of the output set from which the
link matrix was derived, for, as shown previously, to change from one output
set to another, the output is moved around loops, thus moving the outputs only
within these groups.
For the above example (25) and output set of (11), the following Boolean
matrix is obtained"
1 2 3 4 5 6 7
1 0 1 0 0 0 1
0 0
0 1
(27) B= 4 1 0 0
0 1 0
0 0
0 0 1
This matrix raised to the seventh power becomes
1 2 3 4 5 6 7
0 1 0 0 0 1
0 0 0 1
(28) B= 4 0 1 0 1
0 0 0 0
1 0 1 1
0 1 0 0 1
In the matrix B b bl. 1 fori 1, 3, 7. Thus, (1 3, 7) constitutes a

336 D. Vo STEWARD
group of equations. Similarly, the groups of equations (2, 6), (4), and (5) are
obtained.
Next, the precedence relationships among these groups of equations must be
determined. To do this, the "inclusive-OR" of the columns representing equations
in the same group is taken, followed by the "inclusive-OR" of the rows repre-
senting equations in the same group. This process eliminates the chains between
the equations within a group, and leaves simply the chains between groups of
equations. For the example above,
1, 3, 7 2, 6 4 5
1, 3, 7
(29)
This precedence matrix then represents the precedence relationship of the

groups of equations in (26).
7. SUMMARY
The concept of information flow has been introduced for the purpose of study-
ing systems of equations. This concept has been applied to a method for par-
titioning systems of equations for faster solutions on computers with parallel
processing ability. In two appendices, the concept is used to study iteration and
elimination as an indication of the possibilities for further work.
REFERENCES
1. :FRANK HARARY, A Graph Theoretic Method for the Complete Reduction of a Matrix with
a View Toward Finding its Eigenvalues, J. Math. Physics. v. 38, 1959/60, pp.
104-111.
2. SAMUEL MASON, Feedback Theory--Some Properties of Signal Flow Graphs, Proc. IRE,
v. 41, 1953, pp. 1144-1156.
3. J. PAUL ROTH, An Application of Algebraic Topology: Kron’s Method of Tearing, Quart.
Appl. Math. v. 17, 1959, pp. 1-24.
4. MYRON TRIBUS, "On an Aid to System Optimization," Report No. 58GL236, General
Electric Company--General Engineering Laboratory, 1958.
5. HERBERT SIMON, Models of Man: Social and Rational; Mathematical Essays on Ra-
tional Human Behavior in a Social Setting, New York, John Wiley & Sons,
Inc., 1957.
6. DONALD V. STEWARD, On an Algebraic Foundation for Constructing Optimum Algorithms,
Preprints, 14th National Meeting, Assoc. for Comp. Mach., Sept. 1-3, 1959.
APPENDIX A--ITERATION
It has been shown that, to solve the equations in a loop, unknowns must be
carried around the loop. An unknown may be carried as a guessed value to be
iterated, or as a symbolic unknown to be solved for. The latter procedure is

equivalent to elimination. Some of the properties of iteration and elimination
are now studied with the use of paths and loops. Iteration is discussed in this
appendix, elimination in the next. In this appendix, only the properties of linear
-
systems under iteration are considered.
First, consider the iterative properties of a single loop, using the example of a
loop which appears in (8) and the output set (lb, 2c, 3a). The exact value of
x, plus an error e is first substituted into equation 1. Solving around the loop,
a new value, x, Ee, is computed. After n such iterations, x Ene is obtained.
Solving for E indicates that, in general, for a single loop,
(A1) E
out, i]
where a.. and aout,i are the respective coefficients of the input and output
variables of equation i, and where i, in turn, ranges over the equations in the
loop.
If[El < 1, the iteration converges. If]E > 1, the iteration diverges. But
note that, ff the direction of the loop is reversed, the roles of input and output
are reversed, giving the reciprocal of E. Thus, if the original direction of the loop
caused divergence, the reverse direction will cause convergence, and vice versa.
If E -1, the answer is the average of two successive iterations. The case
wherein E 1 is equivalent to the determinant being zero--that is, the answer
(ff it exists) is not unique.
Consider now the case having more than one loop in the set, and in which
iterations proceed on several ariables simultaneously. The following set of
equations illustrates this case.
a a b
(A2) a a acX b
(a a aa ba.
An output set must be assigned, as well as an order in which the equations are
solved. The iterative properties are functions of these assignments. For the ith
equation in the order, all those wfiables not assigned as output in the first i
equations become input from the previous iteration.
For (A2), the diagonal terms will be selected as outputs, and 1, 2, 3 as the
order. Thus, the terms above the diagonal become input from the previous
iteration.
If, for each variable, the current values of the variables plus an error are
substituted, the following is obtained"
a(x + ) + a(x + ) + a(x + ) b
(a(x. + ) + a(x + ) + a(x + ) b,

338 D.V. STEWARD
where the ’s are inputs from the previous iteration, and the e’s are computed
for this iteration. Note that the i’s appear above the diagonal.
Subtracting the original equations, the following is obtained"
-t- albb -t- alcc
alaea 0
(A4) aa -t- a a 0
The ei’s are computed in the present iteration, and are the following functions
of the i’s:
(As)
where
ale
bac
ala
(6)
b =(_a(_a
bc= (--a( a a
b ( ab(/k (_ /k
aib(_ /k-- )
[b (--a(aa) a)(_ a2a( a o a
A set of paths from variable i to variable j can be associated with each by

These paths can be generated as follows: The variable in each equation appearing
on the left (unknown, the present iteration) side of the equation is indicated.
The link matrix for this system is written, and l’s are then placed on the diag-
onal. The definition of multiplication of chains can be used here in the obvious
way to define the multiplication of paths. Multiplication of a chain by 1 is de-
ed as usual (that is, c. 1, 1. c c, where c is any chain). For the example,
the following array indicates the appearance of unknown variables to be solved
for in the present iteration"
a b c
1 X
(A7) 2 x x
3 X X
Let L be the link matrix for this reduced system.

1 2 3
(AS) L 2
,
The set of all chains which occur in this reduced system is given by L where n
is the order of L. Note that L will have only O’s below the diagonal.
Let E be a matrix showing those equations which use the variables computed
in the previous iteration. For the example,
1 2 3
(A9) E b
Let F be a diagonal matrix describing the output set. Thus,

a b c
1 la 0 0
(A10) F 2 0 2b 0
3 0 0 3c
Then, the matrix from which the b,’s can be derived is

(All) [c,,] EL’F.
For the example,
a b c
bla bla2b bla3c

bla2b3c
(A12) [c..]
cla cla2b cla3c
c2b cla2b3c
c2b3c
340 D.V. STEWARD
The element c. contains all of the paths from variable i to variable j. The co-
efficient b. can be obtained as follows:
Z: II
all paths all equations
ain, k
in cii k in the path
where ai,,k is the coefficient of the input variable of equation ]c of the path, and
aout, is the coefficient of the output variable of equation ] of the path. The
input variable appears to the left of the equation in the path; the output variable
appears to the right of the equation in the path.
Consider bb in the example. The paths bla3c and bla2b3c can be displayed
as follows=
(A14)
and
(A15) bc (--
APPENDIX BmELIMINATION
For a single loop, quite clearly, each substitution of a symbolic unknown--
that is, each elimination step--reduces the order of the loop by one. Thus, an
nth order loop may be solved after n 1 substitutions carrying an unknown.
However, in a group of equations with more than one loop, a substitution may
decrease the order of several loops simultaneously, or may even increase the
number of loops. A manner of obtaining the new loop structure, given the loop
structure and a substitution to be made, will be shown.
Let the substitution be represented by evke, meaning that equation e. is
solved for variable v, which is substituted into equation e, to eliminate vk from
e,. Equation e is unchanged by the substitution. Equation e, will now have all
of the variables that appear in either e, or e, except v, which will be eliminated.
It is assumed that only one variable is eliminated at a time.
The following definitions are established"
A, B, C, D a list of entries
(A) a member of a list
(A)’ next entry in a list, if a next entry exists
O(A),
P
substitute e
-
order of (A) (number of variables appearing)
e wherever e. appears
permute loop so that e is first and last element
Pl permute loop so that lowest number e appears as first and last

element
remove element x; where x may be specified as ei, v j, L for last
element, or F for first element
RF remove first n elements
appears in, is a subchMn of
does not appear in, is not a subchain of
logical AND
for all
for some
implies
goes to
set union
Now the rules for the loops resulting from the substitution ejvke on the list of
loops A may be expressed. Each of the conditions of these rules is exclusive of
the conditions on the remaining rules. Let A be the list of original loops. (The
completeness of these rules has not been proved.)
(t1)
(t2)
(R3)
(evke)
(eve)
e ( (A)
(A) ] [0 (A)i
^
(A ] h [O (A
(A)
>
2]
2]
- (B)’
RPe (A) ---. (C)’
h [PS(,)S() (A) # (A) ]

[PtS(,i)S(,i,) (A) (B) ’]
h [RP., (A) (C)’]
(R4) (eve) (A) ] A [0 (A) > 2]
[RP (A) (C)’]

(R5) [e (A)] h [et (A)] A [(eve) (A)] (A)i (B)
(R6) [e (A)] h [e $ (A)] h [S(,i)RPi(A) (A)]
[(A), (B)’] IS (,,i)R Pi (A) (D) ’1
(R7) [e 6 (A),] [e, $ (A),] n [S(.,.)R.P (A), (A) ]
[(A), (S)’]
Mter substitution, the loops are:
BUW,U(C,.D)].
C and D are lists of paths. Multiplication of paths is here extended such that if
342 D.V. STEWARD
the last element of the left path is a variable symbol, that element is not dropped
and the product does exist.
As an example, consider the following system of equations and the substi-
tution 2b1"
a b c d
1 Xq X X
(B 1) 2 X X X X evke 2bl.
3 X X
4 X X
Here,
A B C D Rule
la2bl la2b 2
la4cl la4cl 1
la4c2bl la4c2b 4
(B2) la2b4cl la2b4cl 5
la4c3d2bl la4c3dl la4c3d2b 3
2b4c2 2b4c2 7
2b4c3d2 2b4c3d2 4c3dl 6
After substitution, the loops are
la4cl
1a2b4c1
la4c3dl
(B3) 2b4c2
2b4c3d2
la2b4c3dl,
which is the loop structure of
a b c d
1 X X X
(B4) 2 X X X X
3 X X
4 X X X
ACKNOWLEDGMENTS
This paper is essentially the author’s thesis for the degree of Master of Science,
Department of Mathematics, Stanford University. I would like particularly to
thank Mr. Wesley H. Harker, whose encouragement sparked my enthusiasm
and gained support for this work at the General Electric Company, and Pro-
fessor George E. Forsythe of Stanford University and Dr. Richard W. Hamming
of Bell Telephone Laboratories, my thesis advisors, for much helpful criticism.

Steward 1962

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Steward 1962

Uploaded by

Copyright:

Available Formats

SIAM Rvxw

Vol. 4, No 4, October, 1962

ON AN APPROACH TO TECHNIQUES FOR THE ANALYSIS OF THE

(8) 2 XX chain (loop) lb2c3al

(9) 1 (x, y, z real)

Let i represent the output variable of equation i. Then, ai is defined to be

(16) [a.] 2b3cl 2b3c2 0

output set la, 2b, 3c)

Consider now a formal procedure for the transformation of loops caused by

[--la2--2b3 3cl ---] loop to be transformed

It would be beneficial to partition the solution of large systems of equations

This matrix raised to the seventh power becomes

In the matrix B b bl. 1 fori 1, 3, 7. Thus, (1 3, 7) constitutes a

This precedence matrix then represents the precedence relationship of the

iterated, or as a symbolic unknown to be solved for. The latter procedure is

(a(x. + ) + a(x + ) + a(x + ) b,

A set of paths from variable i to variable j can be associated with each by

Let L be the link matrix for this reduced system.

Let F be a diagonal matrix describing the output set. Thus,

Then, the matrix from which the b,’s can be derived is

bla bla2b bla3c

in cii k in the path

Pl permute loop so that lowest number e appears as first and last

h [PS(,)S() (A) # (A) ]

[RP (A) (C)’]

You might also like