Professional Documents
Culture Documents
Fe7c PDF
Fe7c PDF
algebraic systems” (translated by G. J. Tee), Computer Science Dept., I4O1 R. W. Stineman, “Digital timedomain analysis of systems with
Stanford University, Stanford, Calif., Tech. Rept. CS24, 1965. widely’separated poles,” J. Assoc. Comp. Mach., vol. 12, pp. 286-293,
[291 N. Sato and W. F. Tinney, “Techniques for exploiting the sparsity April 1965.
of the network admittance matrix,’’ IEEE Trans. Power and Apparatus, I4l1 P. I. Richards, W . D. Lanning, and M. D. Torrey, “Numerical
vol. 82, pp. 944-950, December 1963. integration of large, highlydamped, nonlinear systems,” SIAM Rev.. vol.
I 3 O 1 W. F. Tinney and J. W. Walker, “Direct solutions of sparse net- 7, pp. 376380,July 1965.
work equations by optimally orderedtriangularfactorization,”this [421 C. E. Treanor, “A method for the numerical integration of
issue, p. 1801. coupled first-order differential equations with greatly different time con-
S. Parter, “The use of linear graphs in gauss elimination,” SIAM stants,” Math. Comp.,vol. 20, pp. 3945, January 1966.
Rev., vol. 3, pp. 119-130, April 1961. ‘431 E. R. Cohen, “Some topics in reactor kinetics,” Proc. 2nd UN
[ 3 2 1 D. Bree, Jr., “Some remarks on the application of graph theory to Internat’l Con$ on Peaceful Uses of Atomic Energy (Geneva, Switzerland),
the solution of sparse systems of linear equations,” thesis, Dept. of Math., VOI.11, pp. 302-309, 1958.
Princeton University, Princeton, N. J., May 1965. [441 E. R. Cohen and H. P. Flatt. “Numerical solution of quasi-linear
‘’’I A. Orden, “Matrixinversion and related topics by direct methods,” equations,” Proc. Seminar on Codes for Reactor Computations (Vienna,
in Mathematical Methods for Digital Computers, A. Ralston and H. S. Austria), pp. 461484, 1960.
Wilf, Eds. New York: Wiley, 1960, ch. 2. 14’] J. Certaine, “The solution of ordinary differential equations with
[341 F. B. Hildebrand, Introduction to Numerical Analysis. New York: large time constants,” in Mathematical Methods for Digital Computers,
McGraw-Hill, 1956. A. Ralston and H.S. Wilf, Eds. New York: Wiley, 1960, ch. 11.
1351 C. G. Broyden, “A class of methods for solving nonlinear simul- [461 J. Katzenelson, “AEDNET: a simulator tor nonlinear networks,”
taneous equations,” Math. Comp., vol. 19, pp. 577-593, October 1965. Proc. IEEE,vol. 54, pp. 15361552, November 1966.
[361F. H. Branin, Jr., and H.H. Wan& “A fast reliable iteration method [471 G . N. Polozhii, The Method of Summary Representation for
for dc a n a l y s i s of nonlinear networks,” this issue, p. 1819. Numerical Solution of Problems of MathematicalPhysics. New York:
t 3 7 1 J. Katzenelson, “An algorithm for solving nonlinear resistive net- Pergamon, 1965.
works,” Bell Sys. Tech. J., vol. 44, pp. 1605-1620, November 1965. 1481 System Analysis by Digital Computer, F. F. Kuo and J. F. Kaiser,
l3*1 J. G. F. Francis, “The Q-R transformation, pt. 1,” Cornpurer J . , Eds. New York: Wiley, 1966.
vol. 4, pp. 265-271, October, 1961; “The Q-R transformation, pt. 11,” 1491 G. C. Temes and D. A. Calahan,”Computer-aided network
Computer J., vol. 4, pp. 332-345, January 1962. optimization-The state-of-the-art,” this issue, p. 1832.
[391M. L.Liou, “A numerical solution oflinear time-invariant systems,” G. D. Hachtel and R. A. Rehrer, “Techniques for theoptimal
Proc. 3rd Allerton Conf: on Circuit and System Theory, pp. 669-676, 1965. design and synthesis of switching circuits,” this issue, p. 1864.
a*n 1
.
. a2n b,
. . .
an2 . . .
.
ann
.
bn
] (2)
Jordan decomposition. The first step is to divide the elements of the first row
by a, as indicated in (3).
This paper extends previous work and presents it in a
new matrix form. The method is used at Bonneville Power a‘” =
lj (l/all)aIj j = 2, n
Administration (BPA) in many applications including b\” = (l/all)bl. (3)
power short circuit, transient stability, network re-
duction, switching transients,[*] reactive optimization,[g1 The superscripts indicate the order of the derived system.
tower design,[”] and others. The second step, as indicated in (4a) and (4b), is to elim-
inate a,, from the second row by linear combination with
the derived firstrow, and then to divide the remaining
11. FACTORED DIRECT SOLUTIONS
derived elements of the second row by its derived diagonal
This section, covering the first part of the method, shows element.
how to derive an array of numbers from a nonsingular
matrix A that can be used to obtain the effects of any or all
of the following: A, A - ’ , A‘, (Ar)-’, and certain two-way
hybrid combinations of these matrices. The superscript - 1
means inverse and the superscript t means transpose. The
schemeis applicable to any nonsingular matrix, real or
complex, sparse or full, symmetric or nonsymmetric. Al-
though the examples in this paper are limited to nodal
equations, the method is also applicable to mesh equations.
Its greatest advantage isrealized in problems involving
large sparse matrices. The third step, as indicated in (sa) and (5b),is to eliminate
The scheme to be described in this section is similar to elements to the left of the diagonal of the third row and to
those associated with the names of Gauss, Doolittle, divide the remaining derived elements of the row by the
Choleski, Banachiewicz, and others. All of theseclosely derived diagonal element.
related schemes are computational variations of the basic
process of triangularizing a matrix by equivalence trans-
formations. They were originally developed for, and until
very recently have only been described in terms of, manual
computational procedures. Very little attention has been
given to their special suitability for sparse matrices.
The basic scheme is first presented for the most general
case, a full nonsymmetric matrix. Symmetry isthen treated
as a special case. Sparsity with optimal ordering, which is
the primary objective of this development, is explained in
Section 111. Numerical examples of the basic scheme are
given in Appendix I.
A. Triangular Decomposition
Triangular decomposition of a matrix by Gaussian elim-
ination is described in many books on matrix analy~is.[~]*[~]
Ordinarily, the decomposition is accomplished by elimina-
tion of elements below the main diagonal in successive
columns. From the standpoint of computer programming
for a sparse matrix, itis usually much more efficient to
eliminate by successive rows. Therefore, this less familiar,
but mathematically equivalent, scheme is illustrated here.
The development is based on the equation
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.
WALKER:
AND
TINNEY SOLUTIONS EQUATIONS
OF
NETWORK
SPARSE 1803
It should be noted that at the end of the kth step, work The matrix brackets are omitted in (8) to emphasize that
on rows 1 to k has been completed and rows k + 1 to n the array is not strictly a matrix in the same sense as pre-
have not yet entered the process in any way. ceding examples, but only a scheme of recording. It will be
The solution can now be obtained by back substitution. referred to asthe tableof factors. In the literature this result
is frequently shown as a factoring of the inverse matrix
x, = b(") into the productof a lower and an upper triangular matrix,
(a- 1 ) - 0- 1 )
(7)
but it is more suitable for this discussion to consider it as
xn-1 = bn-1 4-1,nXn
a table of factors.
x.I = b!i)- .!:)xj.
1
j=i+ 1
C. Computing Direct Solutions
It is convenient in symbolizing the operations for ob-
In programming, the xi's replace the hi's one by one as taining direction solutions to define some special matrices
they are computed, starting with x, and working backto xl. in terms of the elements of the table of factors (8). The fol-
When A isfull and n large, it can beshown that the lowingnonsingular matrices differfrom the unit matrix
number of multiplication-addition operations for triangular only in the row or column indicated.
decomposition is approximately 1/3n3 compared with n3
for Di: Row i = (0, 0, . . . 0, dii,0, . . . 0,O)
It can easily be verifiedthat triangularizationin the same L ~C : o l i = (O,O,...O,I,-Zi+l,~,-Zi+2,i,...-Z,-l,i ,-Zn,i)t
order by columns instead of rows would have produced L~:Rowi=(-li,l,-li,2,~~~-li,i~l,1,0,~~
identically the same result. Each eliminated element a$-'), Ui:R~wi=(0,0,~~~0,1,-ui,i+1,-ui,i+2,~~~
i > j , would have been the same and the number of opera- U:: Coli=(-ul,i, - u ~ , ~* *, .- u ~ - ~ 1,0, , ~ , **.O,O)f.
tions would have beenthe same. The back substitution also
could have been accomplished by columns instead of rows (10)
in the same number of operations. The inverses of these matrices are trivial. The inverse of
the matrix D iinvolves only the reciprocal of the element
B. Recording the Operations dii. The inverses of the matrices Li,L:, Vi,and U t involve
only a reversal of algebraic signs of the off-diagonal
If the forward operations on b had been recorded so that
elements.
they could be repeated, it is obvious that with this record
The forward and back substitutionoperationson the
and the upper triangle (6) for the back substitution, (1)
column vector b that transform it to x can be expressed as
couldbesolvedforany vector 6 without repeating the
premultiplications by matrices Di, or L :, and U ior U:.
Li
triangularization. The recording of the forward operations,
Thus the solution of Ax = b can be expressed as indicated
however, is trivial. Each forward operation is completely
in ( I l a H l l d ) .
defined by the row and column coordinates and value of
a single element a$-'), i 2 j , that occurs in the process. U1U2... U,-2Un-1DnLn-1Dn-1Ln-2~.~ L2D2LlDlb
Therefore, it is unnecessary to do anything to record these
= A-'b = x ( l l a )
elements except to leave them.
The rules for recording the forward operations of tri- U1U2"' U,-2Un-,D,L,*D,-,L:-,'..L3D2LtD,b
angularization are: = A-'b = X (llb)
UfU3.~.U~-1U,*D,L,-1D,-1L,-2...L2D2LlDlb
1) when a term l/u!;-') is computed, store itin location ii
= A-'b = X (1IC)
2) leave every derived term ag-'), i> j , in the lower
triangle. U,*-,U,*D,L,*D,_,L,*_,... LSD2LZDlb
UtUf...
= A-'b = X. ( l l d )
Since the forward as well as the back substitution opera-
tions are recorded in this scheme, it is no longer necessary
Each of these four equations describes a sequence of
to include the vector b. The final result of triangularizing A
operations on the vector 6 that is equivalent to premulti-
and recording the forward operations is symbolized in (8).
plication by A - ' . For an nth-order system, each equation
u12 u13 . * * "In indicates n multiplications, n2 - n multiplication-additions,
I21 d22 u23 . . . "2, and n additions, excluding, of course, multiplications by
d33 . unity which are only symbolic. This corresponds exactly
131 4 2 . '43,. (8) with the n2 multiplication-additions required for pre-
. . . . .
4 1 4 2 43 . ' dnn multiplication by the inverse. Starting with D l and pro-
ceeding to the left, ( l l a ) describes the forward and back
The elements of (8), defined in terms of the derived sys- substitution operations that would be performed on 6 if it
tems of A in (2H6),are : augmented A during triangularization by columns. Equa-
tion ( l l b ) describes the same result for triangularization by
d.. = rows. Equations (llc) and ( l l d ) describe other sequences of
u . . = a!')
IJ V i < j (9) the same operations giving the same result. Depending on
1..I J = aQ-1)
IJ
i > j . programming techniques, one of these four equivalent se-
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.
1804 PROCEEDINGS OF THE IEEE, NOVEMBER 1967
quences usually will prove to be the most convenient. By using elements from z and g, the composite vector h
Direct solutions of other systems based on A can be ob- is formed.
tained from the table of factors by use of the following two
theorems : h' = (zl, z2, ' ' ' zk, 1, x k + 2 , ' ' * xn). (17)
9 9
The operations can be extended to include certain two- More convenient expressions fromthe standpoint of opera-
way hybrid solutions with the matrix partitioned at any tion and notation can be obtained by noting that products
point desired. Let the hybrid column vector g be defined as of the form UfD,, i > j , are commutative. Thus all of the Di
factors can be grouped into one diagonal matrix
d = (bl, b2, . ' ' , bk, x k + 1, x k + 2 , . ' ' 9 xn)*~ (15)
D = D1D2' . . D,.
If g is given,the unknown fist k elements of x and the k + 1
to nth elements of b can be obtained directly. First it is With this arrangement (22) becomes
necessary to compute an intermediate vector z.
U l U 2 . . . U n - 2 U n - 1...
D UUiU\b
~ - 1 = A-'b = X. (23)
u-1
n-1
u-1
n-2
. . . uk -t12 u-1
k+lDkL?'*'L~D2LfD1g= z* (16) Similar substitutions for the symmetric case can be made
Equation (16) indicates the solution of an upper triangular in the other equations for direct solutions.
system to obtain the first k elements of z and the solution Symmetry also permits a saving of almost one half of the
of an independent lower triangular system to obtain the triangularization operations becauseitisunnecessary to
remaining elements of z. perform any operations to the left of the diagonal in order
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.
WALKER:
AND
TINNEY SOLUTIONS EQUATIONS
NETWORK
OF SPARSE 1805
to obtain the derived elements a$-'), i > j . These elements F. Comparative Advantagesfor a Full Matrix
can be recovered from previously derived elements by the When A is full, the advantages of the factored form of
relationship direct solution are :
a $ - ' ) = (a$-'))a$) i = 2, n ; j = 1, i - 1. (24a) 1) the array of factors can be obtained in one-third the
An alternative procedure avoids a number of multi- number of operations of the inverse
plications equal to those indicated in(24a). In the non- 2) the factored form gives the effect of A and the hybrid
symmetric case each row was normalized by multiplying matrix; the inverse does not.
each term to the right of the diagonal by the reciprocal of The methods are comparable in that:
the diagonal term before proceeding to the nextrow. If
this step is deferred, leaving the terms of a row unnormal- 1) complete solutions normally require the same number
ized until later, the normalizing multiplication and re- of operations
covery of the symmetric term, when needed, can be com- 2) the storage requirement is the same.
bined. Thus after processing row i, each term is left in the The advantages of the inverse are :
form a{:- '), i <j , instead of a$). When the term a$-'), j > i,
is needed in processing row j to be used as a multiplier of 1) complete solutions require only kn operations when
row i in the same way as if row i had been normalized, it is the independent vector has only k nonzero elements
computed from ujj- '). The operations using this alternative 2) under some circumstances the inverse can be modified
procedure are as follows : to reflect changes in the original matrix more easily
than the table of factors.
J k = Jk [(l/U{! 1 - 1 ) bji-
( i 1 )]ai;-
(I 1) i < j ; k = j , ?I. (24b)
The term in brackets, which is a$) in normalized form, re- 111. SPARSITY AND OPTIMAL ORDERING
places ar:- ') as soon as it has been used in (24b).The saving When the matrix to be triangularized is sparse, the order
in operations is trivial for a full matrix, but s i d c a n t for in which rows are processed affects the number of nonzero
a sparse one. terms in the resultant upper triangle.['] If a programming
If the triangularized systemisleft unnormalized, n scheme is used which processes and stores only nonzero
additional operations arerequired for each direct solution. terms, a very great savings in operations and computer
This is significant for a sparse matrix; therefore, normal- memory can be achieved by keeping the table of factors as
ization as described is recommended. sparse as possible. The absolute optimal order of elimina-
tion would result in the least possible terms in the table of
E. Modifications
factors. An efficientalgorithm for determining the absolute
If A is changed, it is necessary to modify the table of optimal order has not been developed, and it appears tobe a
factors to reflect the change. If an element at,,, of A is practical impossibility. However, several effective schemes
changed, elements of the table of factors withrow and have been developedfor determining near-optimal orders.
column subscripts i, j are affected as follows :
1) i f k > m , elements aijwith i = k , j 2 mand i > k , j 2 k A . Schemes for Near-Optimal Ordering
are affected. The inspection algorithms for near-optimal ordering to
2) if k s m elements aij with i z k , j = m and i 2 m , j > m be described are applicable to sparse matrices that aresym-
are affected. metric in pattern of nonzero offdiagonalterms; i.e.,if
Sincemost changes in a rowinvolve a change in the aij is nonzero, then aji also is nonzero but not necessarily
diagonal term, the first case is the more important. The equal to aij.These are matrices that occur most frequently
number of operations required to effect a change in the kth in network problems. From the standpoint of program-
row is approximately 1/3(n- k + l ) 3 .However, any number ming efficiency, the algorithms should be applied before,
of additional changes in the lower rightsubmatrix bounded rather than during, the triangularization. It is assumed in
by row k and column k can be included in the same pass what follows that the matrix rows are originally numbered
without increasing the volume of computations. Changes according to some external criterion and then renumbered
below row k in columns 1 to k - 1 can also be made in the according to the inspection algorithm. Eliminations are
same pass with only a small increase in the volume of com- then performed in ascending sequence of the renumbered
putations. If the rows in which frequent changes will be re- system.
quired can be anticipated, they should be located at the Following are descriptions of three schemes for renum-
bottom of the matrix. bering in near-optimal order. They are listed in increasing
In using the inverse,every change in A affectsall n2 order of programming complexity, execution time, and
elements of A - ' ; therefore, at least n2 operations on A - optimality. '
are required to account for any change in A. Whether the 1) Number the rows according to the number of non-
inverse or the table of factors is the easier to mod* in a zero off-diagonal terms before elimination. In this scheme
particular situation depends on the nature and number of the rows with only one offdiagonal term are numbered
matrix changes required and the scheme of inverse modi- first, those with two terms second, etc., and those with the
fication being used. most terms, last. This scheme does not take into account
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.
1806 PROCEEDINGS OF THE IEEE, NOVEMBER 1967
divisions = n The original matrix, the table of factors, and all indexing
n- 1 tables contain only nonzero elements.
multiplications = s = 1 Ti. (26a) Rows are transferred from the original matrix to a com-
i= 1
pact working rowinwhich elements tothe leftof the
The number of multiplication-additions is dependent on diagonal are eliminated by appropriate linear combination
whether the matrix is symmetric or not. For nonsymmetric withpreviouslyprocessed rows from the partially com-
matrices with symmetric pattern of nonzero off-diagonal pleted table of factors. When work on the row has been
elements : completed, it is added to the table of factors. The actual
n- 1 procedure varies depending on the nature of the applica-
multiplication-additions = 1 r?.
i= 1
(26b) tion. If the matrix is nonsymmetric, the derived elements to
the left of the diagonal must be stored. The arrangement of
For symmetric matrices : the tables depends to some extent on the subsequent needs
n- 1 for direct solutions.
multiplication-additions = 1 (r? + ri)/2 (2612) An example of one possibility for arrangmg the table of
i= 1 factors is indicated in Table I. This represents the final
where ri is the number of nonzero terms to the right of the result for a seventh-order symmetric matrix. The columns
diagonal in row i in the table of factors and s is the total labeled Loc refer to relative addresses in computer memory.
number of such terms. The ith location in the D Table contains the element dii
Determining the operation countfor changing the array and the location in the U Table of the first uij element of
of factors to reflect changes in the original matrix is quite row i. The column of the U Table labeled J contains the
complicated and dependent upon the actual programming column subscript j of uij. Thus row 2 begins in location 3
scheme. In general, if the changes in A begin in row k, of the U Table and contains u 2 3 ,u26, and u2, in locations
(26aH26-c) may be used with summations beginning at k 3 , 4 , and 5 , respectively. Row 3 begins in location 6.
instead of 1 to get an approximate count.
The operations required for computing any of the possible TABLE I
complete direct solutions based on A are : EWLE OF STORAGE A N D INDEXING SCHEMEFOR TABLE
OF FACTORS
I
multiplications or divisions = n
additions = n (264 D Table U Table
~
-:I['+
better than any other known method for the direct solution Dl L: D2 L:
[: J['
of the sparse linear equations that arise in many network
problems. The method opens the way for improvements in
existing applications and offers
possibilities for new appli-
cations thatwould not be feasiblewith any other approach.
1 -; J[' 1 4 1
-;I [ill
17
C
= (31)
= [1]: . (32)
1
The table of factors for A is
Given the hybrid vector g such that
1 1 3
T T T
2 1 1.
gf = (b1, x29 ~ 3 =) (6, 1 9 1 ) . (33)
2 2 (28)
3 3 % the intermediate vector z is obtained by (34), whichis
based on (16) :
With b given, the equation A x = b can be solved for x
using (29), which is basedon (1la): U; Z
[11.
b X
h
6'
9 = (29)
With x given, b can be obtained from (30X whichis The elements b2 and b3 are computed by (36), whichis
based on (12a) : based on (1 8) :
0;' L;' 0; L; '
1 1I[r l - d r
2
1-
U;'
1
-
1
$ 1
b b'
:]
X Z
'I I' 1
-1'
1 = [l;
-1. 1
=
3
9
14
. (36)
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.
LUTIONS
ALKER:
AND TINNEY O F SPARSE EQUATIONS
NETWORK 1809
dii terms:2 x 1000= 2000 floating point words McGraw-Hill, 1965, pp. 398415.
uij terms : 2 x u)oo = 6OOO floating pointwords E. Bodewig, MatrixCalculus. New York: Interscience, 1959, pp.
solution vector: 2 x 1000= 2000 floating point words 101-124.
index tables: 4OOO integer words [’I W. F. Tinney and C. E. Hart, “Power flow solution by Newton’s
- method,” presented at IEEE Winter Power Meeting, New York, N. Y.,
total 14OOO words. 1967.
H. W. Dommel, “A method for solving transient phenomena in
If the matrix were real instead of complex, the time for multi-phase systems,” presented at Power Systems Computation Conf.,
Stockholm, 1966.
optimal ordering would be unaffected, but the time for [91J. Peschon, W. F. Tinney, D. S. F’iercy, 0. J. Tveit, and M. Cuenod,
computing the table of factors and thedirect solutions would “Optimum control of reactive power flow,” presented at IEEE Winter
be reduced to less than 25 percent of that for the complex Power Meeting, New York, N. Y., 1967.
[lo] G.H. Jensen, “Designing self-supporting transmission towers
matrix. The dimensions of tables with factor 2 could be re- with the digital computer,” presented at Power Industry Computer Ap-
duced by one half. plicationsConf., Pittsburgh, Pa.,May 15-17, 1967.
Authorized licensed use limited to: Iowa State University. Downloaded on September 4, 2009 at 14:41 from IEEE Xplore. Restrictions apply.