150

IEEE TRANSACTIONS ON POWER APPARATUS AND

[19] C. C. Gotlieb and S. Kumar, "Semantic clustering of index

terms," J. ACM, vol. 15, pp. 493-513, October 1968.
[20] F. Lucio and M. Sami, "On the decomposition of networks in
minimally inter-connected subnetworks," presented at the
IEEE Internatl. Symp. on Circuit Theory, Miami, Fla.,
December 4-6, 1968.
21] D. V. Steward, "An improved method for tearing large systems," Atomic Power Equipment Dept., General Electric Co.,
San Jose, Calif., 1968.
[22] B. A. Carr6, "Solution of load-flow problems by partitioning
systems into trees," IEEE Trans. Power Apparatus and Systems,
vol. PAS-87, pp. 1931-1938, November 1968.
[23] E. C. Ogbuobiri and S. Linke, "A unified algorithm for load flow
and economic dispatch in electric power systems," Proc.

SYSTEMS,

VOL.

PAS-89,

NO.

1, JANUARY 1970

[26] J. D. C. Little, K. G. Murty, D. W. Sweeney, and C. Karel,

"An algorithm for the traveling salesman problem," Operations

Res., vol. 11, pp. 972-989, November 1963.

[27] J. M. Dobbie, "A survey of search theory," Operations Res.,
vol. 16, pp. 525-537, May/June 1968.
[28] M. Bellmore and G. L. Nemhauser, "The traveling salesman
problem: a survey," Operations Res., vol. 16, pp. 538-558,
May/June 1968.
[29] H. W. Dommel and W. F. Tinney, "Optimal power flow solutions," IEEE Trans. Power Apparatus and Systems, vol. PAS87, pp.1866-1876, October 1968.
[30] R. Larson, C. Wells, and J. Peschon, "Efficient computation of
state estimates using the inverse of the Jacobian matrix," Wolf
Management Services, Palo Alto, Calif., Tech. Memo. 2,
August 1968.
1967 IFAC Symp. (Haifa, Israel).
[24] T. C. Hu, "Decomposition in traveling salesman problems," [31] R. Bellman, "Dynamic programming treatment of the traveling salesman problem," J. ACM, vol. 9, January 1962.
T. J. Watson Research Center, IBM Corp., Yorktown Heights,
[32] C. Berge, The Theory of Graphs and Its Applications. New
N. Y., Research Rept. RC 1527, December 17, 1965.
York: Wiley, 1964.
[25] T. C Hu, "A decomposition algorithm for shortest paths in a
network," Mathematics Research Center, University of Wis- [331 R. G. Busacker and T. Saatv, Finite Graphs and Networks.
New York: McGraw-Hill, 1965.
consin, Madison, Tech. Summary Rept. 804, September 1967.

Dynamic Storage and Retrieval in

Sparsity Programming
E. C. OGBUOBIRI,

Abstract-It is shown that sparsity programming is no more than a

substitution of a higher level microcompiler for a basic microcompiler
in the storage retrieval and processing involving elements of linear
and multidimensional arrays. The substitution property of microcompilers permits the coding of a program first in a natural language
using formal subscripts and then converting the conventional coding
into a sparsity coding after the conventional coding has been fully debugged. This two-stage process not only preserves coding efficiency
but also will generally shorten the overall program debugging time.
It additionally provides for division of labor between the conventional coder and the sparsity coder. A formal list structuring strategy
which has built-in "garbage collection" for sparsity programming is
described in detail. This strategy constitutes a conversion guide
from conventional to sparsity programming.
INTRODUCTION
M ANY industrial problems tax the memory capacity of
existing computers. Although more and more storage
facilities become available with time, it is well beyond doubt also
that problem sizes grow at a faster rate. For a class of problems
whose solution involves the manipulation of elements of large
but sparse arrays (such as matrices), it has been demonstrated
[1] that storage and processing of only the nonzero elements of
such arrays not only promotes efficient utilization of available
space but also enhances the overall speed with which a solution

Paper 69 TP 2-PWR, recommended and approved by the Power

System Engineering Committee of the IEEE Power Group for presentation at the IEEE PICA Conference, Denver, Colo., May 18-21,
1969. Manuscript submitted January 13, 1969; made available for
printing August, 1, 1969.
The author is with the Bonneville Power Adninistration, Portland, Ore. 97208.

MEMBER, IEEE

is obtained. Thus, even when one has plenty of space for a given
class of problems, one should also consider sparsity coding in
preference to a conventional coding as a means of reducing computer time.
The author is not immune to the reservations of prospective
sparsity programmers. The idea is always there. The goal is
around the corner. However the path to it is complicated. The
difficulty with sparsity coding in scientific programming appears
to lie in the following:
1) there is more to it than knowledge of FORTRAN language,
2) the program logic is more involved, and hence more debugging time is expected.
Certain applications require that the creator of a problem or
the inventor of an algorithm play the role of a programmer and
vice versa. But this is an ideal combination of talents. Sometimes
the creator knows just enough conventional programming to be
able to code his problemi in a smaller scale. Would it not be helpful to find a programmer who does not understand the totality of
the program logic but who has the skill to expand the size of the
existing program. This would, in fact, amount to a division of
labor that does not involve any loss in efficiency.
This paper purports to formalize sparsity programming in such
a way that the subject will become less mysterious to most conventional coders. By introducing and emphasizing a systematic
approach, the task of sparsity programming will be much easier
both conceptually and in practice. Although rectangular arrays
are implied in the text, the extension to higher dimensional
arrays is obvious. In the interest of the majority of users, the
FORTRAN language is assumed throughout the text. The works of
Byrnes [2], Randell and Kuehner [3], and Jodeit [4] are very

inspiring.

151

OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

COLUMN 1
A

COLUMN 2

COLUMN

1 EASMi
A (M, 1)

A(1, 1)

All, N)

Fig. 1. FORTRAN convention for storing a rectangular

END ROW 1

|0

M M+1

STARTING
ADDRESSES

END ROW 2

A(M,N)
array.

END ROW M

2M 2M+1

DIAGONALS

OFF-DIACONALS

END OF

BLOCK A

Fig. 2. Chained-list structure for a sparse matrix.

SPARSITY

A system is sparse if the associated matrix is sparse. Consider

rectangular array A of dimension N X M. Let K be the number
of nonzero elements in the matrix A. The figure of merit by
which the sparsity of A is judged is K/NM. For admittance
matrices arising in electric power network, this figure of merit is
typically less than 0.5 percent. This means that if one were processing a vector whose dimension is 200, one would only need to
process one element of the vector each time instead of 200.
Thus instead of storing 200 elements, only one is stored; on the
other hand, instead of processing 200 terms (or expressions), only
one term (or expression) need be processed, hence the savings in
both space and time. Nor is sparsity restricted to electric power
nietworks. The logic modules in a large-scale digital computer are
sparsely connected; so also are most communication networks
such as connections among cities. The matrices arising in the
solution of partial differential equations by the method of
stencils are generally sparse.

STORAGE ORGANIZATION
Definition
A block of memory is a contiguous set of storage locations.
Consider a rectangular matrix A of dimensions M X N. The
standard FORTRAN practice is to store this matrix column by
column in a block of memory named A. The first column is
followed by the second column, and so on, until the N columns are
exhausted (see Fig. 1). Thus the double-subscripted variable A is
stored as a single-subscripted variable. The element aij is interpreted as A (Mj + i - M). Consequently, the expansion rule for
the ordered pair (i,j) denoting (row index, column index) is
(1)
(i,j) = AI + i-M.
This rule is built into the FORTRAN compiler, and the translation
is automatic whenever the ordered pair is encountered in a coding.
If, in a DIMENSION statement, one had coded A (MN) in place of
A (M,N), then the compiler will be perfectly happy if within the
body of the program one codes A (MIJ + i- M) in place of A (i,j).
Surely the coding A (ij) is simpler to use than the coding A (Alj

+ i-M) which involves a preliminary computation. To take advantage of such simplicity we naturally sacrifice flexibility.
Therefore, when the matrix A is sparse and we can no longer
afford to store all the zero elements, we must devise a specialpurpose storage scheme for which we must also know the associated translation for the ordered pair (i,j). This is the essence of
sparsity programming.
In many applications,

it is easier and more natural for an input routine to build a system matrix row by row than to build the

same matrix column by column. Consequently, this paper adopts

the row-by-row convention.
List Structure for a Matrix
In order to construct a list structure for storing and/or
retrieving the elements of a matrix A of dimension M X N, we
propose the following attributes of a rectangular array:

1) the type of the matrix,

2) the number of rows in the matrix (number of columns is an
optional attribute),
3) the starting address of each row,
4) the end address of each row (optional),
5) the diagonal elements,
6) the nonzero off diagonals stored row by row,
7) other subarrays induced by the processing on the matrix,
8) the first available address within the storage block reserved
for the array.
For matrices of dimensions higher than 2, attributes 1), 2), and
5) may be modified to specify the matrix. Attribute 1) has subattributes: symmetry and variable type (logical, integer, real,
complex, double precision, or a combination thereof). The necessity for attribute 4) must be dictated by the program logic. For
instance, if the storage of a nonzero element of a row is preceded
by a search to determine whether that element already exists in
the row, then the end address is identically determined by the
search. The diagonal elements form an attribute because in most
applications these elements are all nonzero and in such applications their positional relationship can be exploited to conserve space and also expedite retrieval. Even in applications
where the diagonals are not a significant attribute, there is always some other positional list which can be maintained in that
space. Attribute 7) is a very important one because in certain
programs partial lists are maintained in the process of a computation. The population of the active elements of these partial lists
varies from run to run, and it would be nice to make all unused
cells of these partial lists available for other use. Where it is known
that a concatenation of such partial lists has a uniform bound,
this fact should also be exploited in the specialized design of the
list structure. For instance, consider partial lists LI and L2 and
suppose that the number of active elements is, respectively, ni
and n2. How ni and n2 may vary in any particular run is unknown
but it is known that for run i, ni + n2 = ki. Then a vector of
dimension ki can be set aside, and the lists Li and L2 are initiated
from opposite ends of the subblock reserved for the vector.
Attribute 8) is used to determine the address where a new
element can be stored.
Fig. 2 illustrates a chained-list structure in which attributes 2),
3), 5), 6), and 8) have been employed. The ith cell in the subblock of starting addresses points to the address within the
block named A, at which the list of attributes of the first element
of the ith row starts. The last attribute of an element is a pointer
that tells where the list of attributes of the next element in the
row starts. For the last element in a row, the last attribute is
zero. One should note, in this example, that there are exactly
three subblocks of A corresponding to the attributes 3), 5), and

6).

The attributes of an element of a matrix, within a given row,

are

1) column index c,
2) value v of the element (optional),
3) address p of the next element in the row.

152

IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, JANUARY

POINTER TO
NEXT

POINTER
TO

ELEMENT

ELEMENT

1970

l l

I cjI V0
I o II
-1
q

(a)
POINTER
TO
ELEMENT

VI

P I

C2
p

(b)
Fig. 3. Format for element storage. (a) Nonterminal element
of row. (b) Terminal element of row.

Thus the ordered triplet (c,v,p) fully describes an- element in a

given row. Clearly, if the value is logical (0 or 1), the attributes
can be reduced to the ordered pair (c,p) which means that the
presence of a 1 in a column is indicated by the entry c. If the
matrix is type complex, then the attribute v calls for two successive attributes vr and vu representing the real and imaginary parts
of v. In this case, an element within a given row is described by
the quadruplet (c,v,,v1,p). That an element terminates a given
row list is indicated by assigning the value zero to p. For simplicity and without loss in generality we suppose that the elements are real numbers. Fig. 3 illustrates the format for the storage of an element whose value is neither complex nor logical.
STORAGE, RETRIEVAL, AND GARBAGE COLLECTION
Storage at Program Initialization Time
Using the basic list structure described in the preceding section, the initialization of a matrix A of dimension M X N at input time is accomplished by the following procedure.
1) Store the starting address of row i in A (i).
2) Store the diagonal element of row i in A (N+i).
3) Store the column index of the first off-diagonal element of
row i at the starting address of that row. Then store the value of
the element in the next location(s). Finally, store the address of
the cell which is two locations ahead in the next location; after
this is done the next cell becomes the starting address for the
attributes of the next element to be stored. For the last element
of the row, follow the recommended format which is illustrated in
Fig. (3).

Available Address and List Modification

If, after the initialization of an array, it is found that an element must be added to a given row, then this modification can be
accomplished by storing the new element in an available space
and chaining it to that row to which it belongs. Let p be an available address in a block named A. Let (cl,vi,0) be the terminal
element of a row to which we wish to add a new element (c2,v2,0).
Let q be the terminal address of the row. Fig. 4 shows how the
chaining of the new element is accomplished.
By the storage organization described and the chaining procedure, it is evident that the list of a given row can start anywhere within the storage block and end anywhere within the
same block.

Retrieval of an Element
In order to retrieve a given element of a working array A, the
following procedure may be employed:

V21

(b)
Fig. 4. List modification. (a) List before addition of new element.
(b) List after addition of new element.

NAA

FAA

Fig. 5. Chaining format for available addresses.

c
q

NAA

I|

NAA

FAA

NAA

FAA

(aI)

ZZIIIlZIEl

(b)
Fig. 6. Inclusion of new available address q in list of available
addresses. (a) Stack before chaining. (b) Stack after chaining.

(a)

FAA

FAA

NAA

(aI)I
(a)

NAA

(b)

Fig. 7. Update of list of available addresses when new element is

created. (a) Before adding new element (C,V,O). (b) After adding
new element (C,V,O).

1) Determine the row in which the element belongs; let it be

row i.
2) Obtain the starting address of the row from A (i).
3) Determine the column in which the element belongs.
4) Search through the chained list of row i until an element is
found whose column attribute is the one desired; the value of
this element can then be fetched.

Garbage Collection
There are some applications in which nonzero elements of an
array become zero in the course of program execution. A procedure by which the space occupied by these elements can be reused as soon as they become available is referred to as condensation or garbage collection. The essence of condensation is to move
active elements to one side of the storage block and the available
addresses to the other side.

OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

c)

COMPLEX

REAL

d)

COMPLEX
b)

1000

1.5

4250

6000

6500

9000

1000

2.0

5000

7000

8000

11000
13000

1000

2.5

5750

8000

9500

1000

3.0

6500

9000

11000

15000

1000

3.5

7250

10000

12500

17000

1000

4.0

8000

1000

14000

19000

For binary matrices such as incidence matrices, further compactification of storage is possible if the (c,p) attribute of each
element in a row is binary coded so that one or more element
attributes can be packed into one full word. However, the retrieval costs for such packing are unduly high in high-level
languages like FORTRAN. In basic machine languages, this reduction in storage space can be achieved without any sacrifice in computation time.

UNSYMMETRIC

SYMMETRIC
REAL

Fig. 8. Memory budget table for square

according to Fig. 2.

arrays

153

PROGRAMMING STRATEGY
The approach to sparsity programming should be a two-stage
process; the first stage is a conventional programming using
formal subscripts while the second stage is a program conversion.

stored

One technique for effecting automatic garbage collection is by

maintaining a chained list of discontinuous available addresses.
The discipline to be used is a last-in-first-out (LIFO) discipline.
This is achieved by placing all available addresses in a pushdown stack, and whenever a new element is to be stored, the
available address on top is selected. Let the address on top of the
stack be defined as the first available address (FAA) and the one
after be next available address (NAA). Let an element be described by the ordered triplet (c,v,p). We can again use the cells
corresponding to the p attributes of active cells for chaining the
available cells. Fig. 5 illustrates the chaining format.
Fig. 6 illustrates the effect of the creation of a new available
address while Fig. 7 shows the effect of the creation of a new nonzero element.
CENTRAL MEMORY BUDGET
The central memory budget can be exactly determined as soon
as the storage organization is designed. Consider the storage
organization of Fig. 2 for a square matrix of dimension N. Let
be the total number of off-diagonal elements of the matrix. The
starting addresses of the rows occupy N cells. Each diagonal element occupies one or two cells according to whether the matrix
is real or complex. For each off-diagonal element, three or four
cells are required according to whether the matrix is real or
complex. For a symmetric matrix, only one half of the offdiagonal elements need be stored. Thus one arrives at the following formulas:
n

unsymmetric matrix
a) the total memory space for the real case is 2N + 3n
b) the total memory space for the complex case is 3N + 4n
symmetric matrix
c) the total memory space for the real case is 2N + 1.5n
d) the total memory space for the complex case is 3N + 2n.
If we let k be the average number of off-diagonal elements per
row, then n = kN. Let CM denote the total central memory requirement. With these definitions, the formulas a)-d) will
become
a) CM = (3k + 2)N
b) CM = (4k + 3)N
(2)
c) CM = (1.5k + 1)N
d) CM = (2k + 3)N.
Fig. 8 shows the values of CM for typical value of N and k of
rectangular arrays. For any value of N, the value of CM can be
computed from Fig. 8 by multiplying the value of CM corresponding to the given k by N/1000.

Conventional Programming Stage in Fortran

At the conventional programming stage, a conventional coding
should be carried out by assigning variable names to each matrix
or vector used in the program and employing regular subscripted
variables in the progrem coding. For efficiency in coding full advantage should be taken of any inherent symmetry or equivalences in the working arrays and in the operations of the problem program. However, for maximum facility in debugging, no
advantage should be taken of sparsity, and any symmetry in
working arrays should only be reflected in the program logic
and not in the storage organization for those arrays. Since sym-

metry in working arrays and equivalence among operations are

part of the formulation of the problem-solution algorithm, there
should be no confusion about the extent of the conventional
programmer's responsibility. For instance, if a matrix is square
and symmetric, the matrix should be stored in its entirety while
the indexing on the matrix elements references only one half of
the matrix. In this case a COMMENT statement should be used to
notify the--sparsity programmer about such -symmetry. Other
notices to the sparsity programmer include vectors whose lengths
depend on the actual dimension of a problem, vectors whose
lengths are invariant, pairs of vectors whose combined length is
bounded and which are simultaneously active, as well as pairs of
arrays which are never simultaneously active.

Program Conversion Stage in Fortran

The Sparsity programmer is responsible for the program conversion stage. The function of this type of programmer is to
convert the program in such a way that large sparse matrices can
be handled. Input material and information to the sparsity programmer or coder include:
1) a fully debugged efficient conventional program,
2) array symmetries,
3) vectors whose lengths depend on the actual problem dimensions at run time (the constant of proportionality should be
given if known),
4) vectors whose lengths do not change with problems,
5) pairs of simultaneously active vectors in class 3) whose combined length is bounded,
6) pairs of arrays which are never simultaneously active.
Using the preceding information, the sparsity coder designs the
storage organization for all arrays. (The list structure should be
similar to that of Fig. 2, but the number of subblocks may vary
from one problem program to another.) With the aid of the design,
a memory budget is made, and the budget may guide a design
modification if necessary. After the design of storage organization,
the program conversion is then carried out routinely on an
interpretive basis.

IEEE TRANSACTIONS ON POWER APPARATUS AND SYSTEMS, JANUARY 1970

154
DIAG

SA

hN+1

6) Do-Loop with Element Retrieval: The conventional coding

OFFD

2Nt1

2N

20000

(a)
C

MAIN PROGRAM
COMMON / BLOCK / A (20000)
COMMON / BLANK / ISA, IDIAG, IOFFO
ISA- i
IDIAG - N #I
IOfFD: 2 N.1

CALL SOLVE IA)

STOP
END
SUBROUTINE SOLVE (B)
DIMENSION B(i)
COMMON / BLANK / ISA, IDIAG, IOFFD

RETURN
END

(b)
Fig. 9. Block design and base address initialization. (a) Block A
of dimension 20 000 and with subblocks SA, DIAG, and OFFD.
(b) Sample coding to illustrate modularity and relocatability.

Program Conversion Guide

1) Array Declaration: All arrays whose size depend on the actual
problem dimensions should be assigned to one block with possibly
many subblocks. The block is dimensioned in the main program
in accordance with the memory budget. If the program consists
of a large number of subroutines, the block name should be a
formal parameter in subroutine calls. This means that whenever
there is a change in the memory budget for some reason, only the
DIMENSION statement in the main program need be changed (Fig.
9).

2) Modularity and Relocatability of Subblocks: Since the base

address of the subblocks is in general a function of the problem
size at run time, the base address of each subblock should be
computed and placed in COMMON storage. This means that the
creation and deletion of subblocks do not result in excessive recoding within the body of the program (Fig. 9).
3) Definition of an Array Element: The conventional coding
A (I,J) = X states that the value of X should be stored in the
location of the element a,j of the A matrix. In statements like
this, we should first determine whether the element aij has been
previously defined; if not we should chain the element (J,X,0) to
the list of the Ith row, provided X 5z# 0; if it has been defined as
(J,Y,P), then we simply alter y to x to obtain (J,X,P). If I = J,
we simply code A (IDIAG + I - 1) = X, with IDIAG as the base
address for the diagonal subblock. If X = 0, the element (J, Y,P)
is deleted from the list of row I.
4) Retrieval of an Array Element: The conventional coding X =
A (I,J) states that the value of A (I,J) should be stored in the
location named X. The first step is to search down the Ith row
of A and find an element with attributes (J,-, ). The value
attribute of this element is then stored in X. If there is no such
element in the row list, then X assumes the value zero.
5) Do-Loop with Element Definition: The conventional coding
DO label 1
DO label 2

label 2
label 1

A(I,J)

=
=

ml,M2,M2
ni,n2,n3

CONTINUE

CONTINUE

calls for a direct translation, in accordance with 3), of the statement A (I,J) = X.

DO label 1 I = MIM2,M3
DO label 2 J = ni,n2,n3
X = A(I,J)

label 2 CONTINUE
label 1 CONTINUE

calls for the following procedure:

a) the row-indexing DO should be retained,
b) the column-indexing DO should be removed.
c) each element of row I is checked for the admissible column
index; if the column index is admissible, X is assigned the
value of A (I,J), if not X is assigned the value of zero.
7) Logical Statements: The conventional coding "IF (expression)
labels" where the expression contains A (I,J) as an argument must
be treated as the retrieval statement of 4) and 6). For example,
"IF(A(I,J))1,2,3" should be treated as two consecutive statements:
X = A (I,J)
IF(X)1,2,3.
GENERALIZATION
The basic building block for storage-organization design is the
vector. There is, therefore, no question regarding the manner in
which extension can be made to higher dimensional arrays that
are sparse. Such higher dimensional arrays are simply a vector
of vectors which can be represented in a tree structure. However,
a sparsity program must make a distinction between two types of
vectors: there is the ordinary list in which each member has no
positional attribute; there is also a list of elements in which each
member has a positional attribute. The attributes of the ordinary
list are the base address of its subblock and the number of elements in the list. In this form the elements of the list can be
stored contiguously so as to save the space normally occupied by
positional indices. An example of an indexed list is the list of nonzero elements of a row (or column) of a matrix. An example of a
nonindexed list is a list of the rows of a matrix which are equal.
Whenever possible nonindexed lists must be stored without explicit chaining.
PROGRAMMING EXPERIENCE
The foregoing procedure for sparsity programming has been
applied to a FORTRAN program containing over ten pages of conventional FORTRAN coding and eight subroutines. A successful
conversion took 8 man hours. In this experiment, the conventional coder and the sparsity coder are one and the same person.
It was recognized
1) that the debugging time for the conventional coding was
short because of the simplicity in coding provided by the automatic array handling (by FORTRAN compiler) via the formal subscripts,
2) that the speed of program conversion from a conventional
coding to a sparsity coding is not affected by the division of labor
between two individuals by virtue of the fact that the conversion
procedure is fairly routine,
3) that the efficiency of the sparsity coding in terms of
sparsity conservation is a function of the ability of the conventional coder to provide all of the necessary interface information described in this text; the loss in efficiency is, however, not
critical.
4) the routine conversion procedure is amenable to automation via software implementation.

OGBUOBIRI: DYNAMIC STORAGE AND RETRIEVAL IN SPARSITY PROGRAMMING

CONCLUSION
In outlining the essence of sparsity programming in a problemindependent context, the complete separability between the
functions of a conventional coder and those of a sparsity coder
has been demonstrated and emphasized. The need for this separability of tasks becomes more evident in large complex programs and is not diminished by the fact that the two coders may
be one and the same person since the overall programming
efficiency and debugging speed are enhanced by the staging of the
two tasks. The method of storage and retrieval of array elements
has been formalized in a manner providing automatic garbage
collection and a rule-of-thumb calculation of core-storage requirements; sample formulas and memory budget tables are
given. The examples of the recompiling techniques for the key
conventional codings constitute a program conversion guide;
this conversion guide lends itself to a software implementation.
REFERENCES
[1] W. F. Tinney and C. E. Hart, "Power flow solution by Newton's

method," IEEE Trans. Power Apparatus and Systems, vol.

PAS-86, pp. 1449-1460, November 1967.
[2] F. D. Byrnes, "A program for direct solution of matrix equations," Bonneville Power Admin., Portland, Ore., Rept., June
1967.
B.
Randell and C. J. Kuehner, "Dynamic storage allocation
(3]
systems," Commun. ACM, vol. 11, pp. 297-306, May 1968.
[41 J. G. Jodeit, "Storage organization in programming systems,"
Commun. ACM, vol. 11, pp. 741-746, November 1968.

Discussion
Jen Hsu (P.O. Box 361, Berkeley, Calif. 94701): The paper essentially
pointed out the utilization of linked lists to minimize storage for
sparsity matrices. Linked lists, a well-known method in programming
technique [5], are composed of items which do not usually occupy
contiguous locations in memory. Therefore, an item contains secondary information, i.e., a pointer to where the next item in the chain is
located.
For more effective programming, I would suggest the use of a
circular list instead of the linear list which the author described. We
know that a circular list can be used not only to represent an inherently circular structure but also to represent a linear structure,
i.e., a circular list with one pointer to the rear node. Although a circular list is essentially equivalent to a straight linear list with two

Manuscript received June 12, 1969.

155

pointers to the front and rear, it will be possible to have access to all
of the list starting at any given point, and it will be more effective to
move the leftmost to the right [5].
REFERENCES
[5] W. D. Maurser, Programming, Introduction to Computer Lauguages
and Techniques. San Francisco: Holden-Day, 1968.

E. C. Ogbuobiri: The answer to Mr. Hsu's remarks is in the paper itself. However, the following additional comments are offered.
The paper claims that there is a simple methodology for sparsity
programming and describes this methodology in detail. It is emphasized, even in the abstract, that this methodology involves no more
than a replacement of the FORTRAN storage convention for multidimensional arrays by special-purpose storage organizations. For a
class of problems, Gaussian elimination, for instance, the storage organization must be designed prior to translation into sparsity code only
after the conventional code, which assumes arrays with no sparsity,
is fully debugged. A concrete general-purpose demonstration is given
for the simple problem of storage and retrieval of elements of real
unsymmetric arrays together with a method for garbage collection.
The paper also claims that the two-stage approach described fosters
division of labor and also enhances overall gains in programming
efficiency.
The design of storage organization-takes into account Mr. Hsu's
observation. An array is stored as a tree of lists. The terminal lists
are linear. Hence a vector becomes the building block of an array.
A vector together with the use (and the sequence of creation and extinction) of its elements in the course of computation determine the
most efficient list structure. While some I/O buffer may lend itself to
a circular list, most storage and retrieval problems of numerical
computation exploiting sparsity do not. While, in Gaussian elimination, implicit chaining of row elements is always possible, rows of
certain incidence matrices whose row elements arrive randomly at
input time require explicit chaining. Thus there is room, in each
problem, to optimize the storage organization. However, if the programmer were simply to implement a general-purpose storage and
retrieval scheme for multidimensional arrays without due regard to
numerical operations-on the array elements, as is done in IBM System/360 MATLAN (Qctober 1968 edition) he will still achieve overall
gains in both storage space and computation time for reasons givein
in the paper; the bottleneck Droblem of memory requirements will at
least be solved. He can always improve on the gains by improved
storage organization that takes into account the process of computations to which an array is subjected.
It should be self-evident that this author is not selling a particular
list structure (since the most efficient list can be found for each specific
application) but a methodology which is an outgrowth of his experience in sparsity programming. If any user understands -and -applies
this methodology in his problems, his sparsity code and programming effort in a given computer language can, in that language,
only be equaled but not excelled.

Manuscript received by June 27, 1969.