Professional Documents
Culture Documents
AbstractThis paper presents the solution approach and im- Diakoptics over Sparsity in terms of solution times, and while
plementation strategy followed in the development of UBCs regular or modified nodal analysis with sparsity are straight
OVNI (Object Virtual Network Integrator) power system forward to understand and to implement, Diakoptics, in its
simulator. OVNIs aim is to simulate full-size power system original form, requires obscure manipulations of Contour
networks for real-time and online dynamic studies. To cope
Theory of Networks. To achieve computational efficiency,
with the dimensions of the problem, partitioning techniques
are used, both in the system solution and in the hardware both Diakoptics and Sparsity take advantage of the very
implementation. The software solution is based on the MATE sparse nature of the admittance matrix (many off-diagonal
(Multi-Area Thvnin Equivalent) network partitioning zeroes) of power networks (and most other electrical net-
framework, while the hardware implementation is realized works). Diakoptics exploits this sparsity by splitting the net-
using a cluster of interconnected PC computers. work into dense subsystems connected by a few links. Spar-
Keywords: Real-time OVNI simulator, the MATE algo- sity techniques simply avoid storing and operating with the
rithm, network partitioning for large systems, diakoptics, zeroes of the full system matrix.
EMTP solution, PC-clusters, distributed computing. It has been, however, in the context of real-time simula-
I. INTRODUCTION tors [6], and particularly in the case of multi-computer solu-
tions ([8]), that the advantages of Diakoptics over Sparsity
It was Gabriel Kron in the 1950s who introduced the have become strongly manifested. While both Diakoptics
concept of Diakoptics (from the Greek dia for very and and Sparsity take advantage of the sparsity of the system,
kopto for tearing) for the solution of large power sys- Diakoptics maintains the identity of the component subsys-
tem networks (see, for example [1]). In his comprehensive tems, something that is very difficult to achieve in solutions
review of Diakoptics in [2] in 1974, Happ already visualized based on matrix reduction with Sparsity techniques. Main-
its potential in multiple-computer hardware configurations. taining the individuality of the subsystems permits, for ex-
In [2], Happ proposes the concept of a cluster of comput- ample, the pre-calculation (before the simulation loop) of
ers with each computer in the cluster solving a separate the inverse solution matrices for those subsystems which
part of the whole network. topology does not change [6], while only some of the sub-
Except for some notorious examples mostly due to Happ systems require recalculation. This aspect alone gives Dia-
(see list of references quoted in [2]), Diakoptics did not fulfil koptics a considerable advantage over Sparsity Techniques.
its potential of becoming a universal tool for the study of Other (not less important) advantages of maintaining the
large and complex power systems. Two factors determined individuality of the subsystems, such as latency exploita-
Diakoptics fate in the following fifty years. The main one tion, will be discussed later in this paper.
was the success of Sparsity Techniques pioneered by Tinney UBCs OVNI real-time simulator is built around the MATE
and Walker [3] and advanced by many others. The other (Multi-Area Thvnin Equivalent) concept which extends
aspect was the difficulty of visualizing and formulating the the main ideas of Diakoptics in a conceptually clear and
Diakoptics solution in a clear and straight forward manner. computationally efficient solution framework. MATE goes
In 1975, Ho, Ruehli, and Brennan published their seminal beyond Krons Diakoptics and Ho, et al.s MNA in that it
paper [5] on Modified Nodal Analysis (MNA). MNA sepa- recognizes that the subsystems split by the branch links
rates the network solution into nodal equations and branch can be represented by Thvnin Equivalents. This becomes
equations. Their main motivation was to combine current- a key point when combining together, in a simultaneous
dependent elements, which are much easier defined in terms non-iterative solution, subsystems that have been solved
of branch equations, with the nodal equations for the rest of with different solution techniques (for example with differ-
the network, thus maintaining the efficiency of the nodal ent-size Dts). The hardware implementation of MATE is
solution using sparsity techniques. Even though the au- realized in a multi-machine PC-Cluster environment.
thors did not relate their work to Diakoptics, the resulting
MNA equations are the same as the Diakoptics equations, II. MATE (MULTI-AREA THVNIN EQUIVALENT)
with the augmented branch equations corresponding to
The MATE concept [7] establishes a software/hardware
Diakoptics tearing links.
solution framework around which UBCs OVNI Real-Time
The main aim of Krons Diakoptics was to achieve in- Simulator is designed. MATE embodies the concepts of
creased computational speeds. With the development of Multinode Thvnin Equivalents [4], Diakoptics [1], and Ho,
Sparsity techniques, however, the fact is that on normal off- et al.s Modified Nodal Analysis [5] in a simple and general
line single-computer solutions there is little advantage of formulation. The main idea in MATE, as in Diakoptics, is
This paper is provided on a personal basis only for the private non-commercial use of the requester for educational or
1 restrictions.
research purposes and is subjected to the publisher's copyright
that a large network can be split into smaller parts (subsys- hA2
gA20 A2
tems) by tearing apart some of its branches. Ho, et al.s modi- hB2
fied nodal analysis is used to combine, on the same system B2
gA12
matrix, nodal equations for the individual subsystems with gA23
a gB12 gB20
A1 A2 A3 A4 B1 B2 B3 a1 a2
A1 GA11 GA12 0 GA14 0 0 vA1 hA1
A2 GA12 GA22 GA23 0 0 0 vA2 hA2
A3 0 GA23 GA33 GA34 1 0 vA3 hA3
A4 GA14 0 GA34 GA44 0 1 vA4 hA4
= (4)
B1 GB11 GB12 GB13 -1 0 vB1 hB1
B2 GB12 GB22 GB23 0 0 vB 2 hB2
B3 GB13 GB23 GB33 0 -1 vB3 hB3
a1 0 0 1 0 -1 0 0 - za1 0 ia1 0
a2 0 0 0 1 0 0 -1 0 - za 2 ia 2 0
2
tween subnetworks. Expanding the submatrices and intro-
ducing the links, we get the MATE matrix of (4). vA
-1 -1
The MATE matrix of (4) is built as follows. GA11 = gA12 + A A 0 A p vB = A -1 h A (7b)
gA14, GA12 = -gA12, etc. The empty areas in the matrix corre- ia
spond to zero elements. The link branches are hooked up
as follows. Since current ia1 leaves node A3, a 1 is put in vA
position A3 of column a1 and a 1 is put in position A3 of 1 0 a vB = eA (7c)
row a1. Since current a1 enters into node B1, a -1 is put in ia
the corresponding positions of column a2 and row a2. Simi-
larly for link current ia2. If, in a more general case, branches We apply B-1 to row B:
a1 and a2 were coupled through a z12 impedance, this im-
pedance would occupy the off-diagonal positions of the a vA
subblock. B -1 0 B q v B = B -1 hB (8a)
After introducing the link equations, the MATE matrix ia
(4) becomes the hybrid system of modified nodal analysis
[5] containing nodal equations for the subsystems node
vA
voltages and branch equations for the link currents flowing -1 -1
between subsystems. Equation (4) also corresponds to the 0 B B B q v B = B -1hB (8b)
so-called fundamental equation of diakoptics ([9], p. 137), ia
except that (4) is written directly from nodal analysis and
branch currents while diakoptics requires decomposition into vA
primitive networks and connectivity matrices.
0 1 b v B = eB (8c)
We will now deviate from Diakoptics and from modified
nodal analysis in the way (4) is solved. Our main aim will be ia
to manipulate the submatrices in (4) in a form that preserves
The last row in (5) represents the link branches,
the individuality of the subsystems and reduces them to
Thvnin equivalents. vA
p t
q t
-z vB = 0 (9a)
We first rewrite (4) using partitioned matrices,
ia
A B a
But, [vA] and [vB] are determined in (7c) and (8c) in terms
A A 0 p vA hA
= (5) of [ia], [eA] and [eB]. Substituting in (9a),
B 0 B q vB hB
a pt qt -z ia 0 e A - aia
t t
p q -z eB - bia = 0 (9b)
Here, ia
0 0
0 - 1 0
0 , q = 0
p= 0 and augmenting with [vA] and [vB] to recover the original
+ 1 0 (6) set of variables,
0 - 1
0 + 1 vA
t t
are current injection matrices derived from the link branches 0 0 p a+q b+z vB = p t e A + q t e B (9c)
hooks (+1 if current comes out, -1 if current goes in). ia
We now operate on (5), one partitioned row at a time. Putting the results of (7c), (8c), and (9c) together, we
obtain
We apply A-1 to row A: A B a
vA A 1 0 a vA eA
-1
=
A A 0 p v B = A -1 h A (7a) B 0 1 b vB eB (10)
ia a 0 0 Za ia ea
where,
3
A1 A2 A3 A4 B1 B2 B3 a1 a2
A1 1 a13 a14 v A1 eA1
A2 1 a23 a24 v A2 eA2
A3 1 a33 a34 v A3 eA3
A4 1 a43 a44 v A4 eA4
= (11)
B1 1 - b11 - b13 vB1 eB1
B2 1 - b21 - b23 vB 2 eB2
B3 1 - b31 - b33 vB 3 eB3
a1 0 0 0 0 0 0 0 Z11 Z12 ia 1 ea 1
a2 0 0 0 0 0 0 0 Z12 Z 22 ia 2 ea 2
4
APPLICATION EXAMPLES work with different Dts in different subparts of it is termed
latency exploitation.
Switching Operations As discussed in [12], MATE provides an ideal framework
for latency exploitation. Suppose, for example, that in the
MATE links concept is particularly advantageous to rep-
network of Fig. 1, subsystem [A] is fast and subsystem
resent switching operations in branches connecting sub-
[B] is slow. Then the independent solution of [A] could
systems. For example, in traditional EMTP modelling, when
be performed with a small Dt, while the independent solu-
an ideal switch closes, the two nodes connected by the
tion of [B] is performed with a large Dt. It is only for the link
switch collapse into a single node. Similarly, when the switch
currents solution and for the updating of the susbsystems
opens two nodes are created out of one. These topological
after the link currents are calculated that care must be exer-
changes slow down the solution in two fronts: i) The change
cised. While the fast subsystem is being solved at the faster
in node subscripts, and ii) The need to retriangularize the
pace dictated by its small Dt, the slow subsystem remains
entire system.
dormant (latent) for the length of its large Dt, except for
Suppose in MATE that subnetworks [A] and [B] in Fig. 1
the updating of its Thvnin equivalent circuit at the link
remain topologically constant during the time loop simula-
nodes. The updating of the Thvnin equivalent of the slow
tion (i.e., only the history sources change from time step to
subsystem is done by an interpolation process between two
time step), and that all switching operations occur in the link
consecutive large Dt steps of its Thvnin Equivalent and
branches. Suppose branch a1 is an ideal switch. If the switch does not require the solution of the internal details of the
is closed, element za1 in the links subsystem of (4) will be slow subsystem during the small Dt steps of the fast sub-
zero, with no consequences to MATEs solution process. If system solution. An upcoming paper by F. A. Moreira and J.
the switch is open, the condition will be ia1 = 0, with the R. Mart will present a more detailed explanation of this proc-
topological effect of eliminating row and column a1 from (4). ess.
Even though there is now a topological change, its effect is
limited to the links subsystem and submatrices [A-1] and Other Applications
[B-1] do not need to be retriangularized. MATEs Thvnin equivalent circuit of Fig. 2 provides a
Reference [10] describes the development of a model for common system of coordinates for the resolution of inter-
a 24-valve HVDC converter for the OVNI simulator using actions among different subsystems. Each subsystem can
MATE solution concepts. A comparison of timings between be first solved independently from the others with its own
this model and a standard EMTP solution (using the Micro- solution technique. The interactions among subsystems can
tran program [4]) resulted in a ratio of 40 times faster for the then be resolved at the level of their Thvnin equivalents.
MATE model. This test was run on a single PC computer Once the link currents are calculated they can then be in-
and OVNIs timing advantage was due to avoiding retrian- jected as current sources into the particular subsystems to
gularization of the solution matrices each time there is a take into account (according to the subsystems own inter-
change in a valves conducting state. nal rules) the effect of its neighbours.
In addition to the examples cited above, other mixed-so-
Latency Exploitation lution environments could include: a) Subsystems solved
In EMTP type of solutions, the same discretization step with different integration techniques. For example, if all im-
size Dt is used for all parts of the network. This is often portant information in subsystem [B] in Fig. 1 is 60-Hz, and
wasteful of computational resources because the common it can be adequately solved using a quasi-steady-state
Dt size is determined by the solution needs of the fastest phasor solution (as traditionally done in machine stability
subsystem in the network, while this Dt may be unnecessar- studies), while subsystem [A] is, for example, an HVDC con-
ily small for other subsystems. A typical example is the case verter solved using a discrete-time EMTP solution. b) An-
of HVDC converters and FACTS devices where the firing other common application where MATE results in consider-
angle needs of the valves may require a small Dt resolution, able savings is, for example, when subnetwork [A] is linear
while the rest of the network can be solved with a larger Dt. and subnetwork [B] is nonlinear. In this case since [A-1]
Another common situation is the modelling of transmis- does not change, the solution of [A] requires minimum up-
sion lines. An important new line model, the z-Line Model dating work, and the most computational work is concen-
([11]), solves the problem of frequency dependent transfor- trated in subsystem [B]. Further savings can be achieved if
mation matrices by subdividing, in length, the lossy part of [B] is modelled as a finite nonlinearity (e.g., as piecewise
the model. This model requires internal Dts equal to the linear), in which case all possible states of [B] can be pre-
travelling time of the total line length divided by the number calculated and pre-stored.
of segments into which the losses are distributed. The full
advantages of this model are realized when the Dt for the III. PC-CLUSTER HARDWARE REALIZATION
internal line solution can be much smaller than the Dt of the The PC-cluster hardware architecture for the OVNI simu-
external system solution. The capability of solving a net- lator has been conceived in two different environments: a
hard real-time environment and a soft real-time environ-
5
ment, see Fig. 3. This organization of hardware resources
aims to match the concepts of MATE described in this pa-
per and of Latency [12], in the most flexible and efficient Soft-Real-Time
Environment
way. On a first-level description, the hard real-time environ-
ment is conceived as a high-performance synchronous ar-
ray of solver nodes dedicated to the task of solving MATEs
Real Time Message
decoupled subsystems. It is also at this level that interac- Passing Interface
tion with external analog devices, such as protective relays Cluster
6
soft real-time simulator. For the rest of the time each envi- ware and hardware in OVNI have been designed for easy
ronment computes the solution separately, with the global maintenance and upgradability. The software has been writ-
time as the only synchronization reference. A standard Mes- ten in C++ using high-level object-oriented design tech-
sage Passing Interface provides flexibility and compatibility niques to allow for the addition of new modules or the up-
across several computing platforms. Using a fine-grain glo- grading of existing ones with a minimum of modifications to
bal clock based on a master-slave scheme, the synchroniza- the base code. The hardware of the PC-Cluster design uses
tion among host clocks can be achieved. The global clock standard off-the-shelf components and allows for easy ex-
for the soft real-time environment is integrated with the indi- pansion with additional units and easy replacement of exist-
vidual CPU schedulers on each node. On each node, the ing ones with a minimum of system retuning.
scheduler provides bounded-jitter time-based scheduling
of computing tasks and communicating channel tasks. At V. REFERENCES
initialization, the global clock module is installed and estab-
[1] Gabriel Kron, Tensorial Analysis of Integrated Trans-
lishes a synchronized time frame. Once a global time has mission Systems, Part III - The Primitive Division, AIEE
been established, the individual scheduler is installed. The Transactions, Vol. 71, Pt. 3, pp. 814-821, 1952.
time-based individual scheduler in the soft real-time envi-
[2] Harvey H. Happ, Diakoptics - The Solution of System
ronment uses the global time for all dispatching decisions. Problems by Tearing, Proceedings of the IEEE, Vol. 62,
The soft real-time environment can also be used to run No. 7, pp. 930-940, July 1974.
analysis application tools on top of the real-time simulator. [3] W. F. Tinney, J. W. Walker, Direct Solution of Sparse
In this case one or more nodes of the soft-real-time environ- Network Equations by Optimally Ordered Triangular
ment can be assigned to process data generated in real-time Factorization, Proceedings IEEE, Vol. 55, pp. 1801-1809,
by the hard and soft-real-time environments. November 1967.
[4] Hermann W. Dommel, EMTP Theory Book, 2nd ed.,
IV. CONCLUSIONS Microtran Power System Analysis Corporation, Van-
The large computational power available today at low couver, British Columbia, Canada, pp. 12-3 to 12-9, 1992-
prices opens a number of opportunities in what can be 1996.
achieved in system simulation. At the same time, as the goals [5] C. W. Ho, A. E. Ruehli, P. A. Brennan, The Modified
become more ambitious (e.g., real time simulation), new so- Nodal Approach to Network Analysis, IEEE Transac-
lution approaches are needed to harness this power. OVNI tions Circuits and Systems, CAS-22, pp. 504-509, June
1975.
(for Object Virtual Network Integrator) is an ambitious project
that aims at attaining real-time solution speeds for large power [6] J. R. Mart, L. R. Linares, Real-Time EMTP-based tran-
system networks using desktop computers. OVNI proposes sients simulation, IEEE Transactions on Power Sys-
tems, Vol. 9, No. 3, pp. 1309-1317, August 1994.
a hardware-mapped software solution strategy based on the
MATE (Multi-Area Thvnin Equivalent) concept for net- [7] J. R. Mart, L. R. Linares, J. Calvio, H. W. Dommel, J.
work partitioning. MATE splits the network into subsys- Lin, OVNI: An object approach to real-time power sys-
tem simulators, Proceedings of the 1998 International
tems which are first solved independently and then their
Conference on Power System Technology,
solutions combined at the level of their Thvnin equiva- Powercon98, Beijing, China, August 18-21, 1998, pp.
lents. By choosing the subsystems to be dense, except for a 977-981.
few interconnecting links, MATE achieves the advantages
[8] J. R. Mart, J. A. Hollman, J. Calvio-Fraga, Implemen-
of sparsity without sacrificing the identity of the subsys- tation of a Real-Time Distributed Network Simulator
tems. By maintaining the identity of the subsystems and with PC-Clusters, PARELEC2000 IEEE Computer So-
solving each subsystem separately, each subsystem solu- ciety Proceedings, Trois-Rivires, Qubec, August 27-
tion can be optimized by using the solution technique that 30, 2000, 5 journal pages.
best suites that subsystem characteristics, for example, dif- [9] A. Brameller, M. N. John, M. R. Scott, Practical Dia-
ferent integration step sizes, different integration rules, koptics for Electrical Networks, Chapman & Hall Ltd.,
phasor versus EMTP solution, etc. An exact simultaneous 1969.
solution to evaluate the link currents among subsystems is [10] S. Acevedo, L. R. Linares, J. R. Mart, Y. Fujimoto, Ef-
then performed at the level of the subsystems Thvnin ficient HVDC converter model for real time transients
equivalents. Since a given subsystem sees the others only simulation, IEEE Transactions on Power Systems, Vol.
through this multi-Thvnin equivalent, the updating of the 14, No. 1, pp. 166-171, February 1999.
internal details within the subsystems is up to the solution [11] F. Castellanos, J. R. Mart, "Full frequency-dependent
needs of the particular subsystem. phase-domain transmission line model," IEEE Trans-
A hardware architecture that mimics the MATE algorithm actions on Power Systems, Vol. 12, No. 3, pp. 1331-
in its partitioning of the full system into solution nodes has 1339, August 1997.
been implemented in the form of a hard real-time environ- [12] F. A. Moreira, J. A. Hollman, L. R. Linares, J. R. Mart,
ment using a cluster of PC computers. The extension of this Network Decoupling by Latency Exploitation and Dis-
cluster to better serve asynchronous events in a soft real- tributed Hardware Architecture, Transactions of In-
time environment is currently under development. Both soft- ternational Conference on Power Systems Transients,
IPST2001, Rio de Janeiro, Brazil, June 24-28, 2001.