Computers and Structures 73 (1999) 545±564

www.elsevier.com/locate/compstruc

Parallel simulated annealing for structural optimization

J.P.B. Leite, B.H.V. Topping*
Department of Mechanical Engineering, Heriot±Watt University, Riccarton, Edinburgh, EH14 4AS, UK
Received 1 October 1996; accepted 12 June 1998

Abstract

The simulated annealing (SA) algorithm has proven to be a good technique for solving dicult combinatorial
optimization problems. In engineering optimization the SA has emerged as an alternative tool to solve problems
which are dicult to solve by conventional mathematical programming techniques. The algorithm's major
disadvantage is that solving a complex system may be an extremely slow convergence process, using much more
processor time than some conventional algorithms. Consequently, simulated annealing has not been widely accepted
as an optimization algorithm for engineering problems. Attempts have been made to improve the performance of
the algorithm either by reducing the annealing length or changing the generation and the acceptance mechanisms.
However, these faster schemes, in general, do not inherit the SA properties of escaping from local minima. A more
ecient way to reduce the processor time and make the SA a more attractive solution for engineering problems is to
add parallelism. However, the implementation and eciency of parallel SA models are in general problem
dependent. Thus, this paper considers the evaluation of parallel schemes for engineering problems where the
solution spaces may be very complex and highly constrained and function evaluations vary from medium to high
cost. In addition, this paper provides guidelines for the selection of appropriate schemes for engineering problems.
An engineering problem with relatively low ®tness evaluation cost and strong time constraint was used to
demonstrate the lower bounds of applicability of parallel schemes. # 1999 Civil-Comp Ltd and Elsevier Science
Ltd. All rights reserved.

1. Introduction algorithm is motivated by an analogy to the behaviour

During the past three decades, there has been a
growing interest in problem solving algorithms inspired
by natural systems in physics and biology. An import-
ant application for these algorithms is to solve combi-
natorial optimization problems, i.e. the process of
®nding one or more optimal or near optimal solutions
that minimize an objective function associated with the
solution. The best known algorithms in this class
include genetic algorithms and simulated annealing.
While the genetic algorithm (GA) attempts to simulate
the nature's genetic breeding, the simulated annealing
* Corresponding author.
of a physical system in a heat bath. Both methods are
able to process functions with arbitrary degrees of
nonlinearities, discontinuities and stochasticity. They
are also able to handle discrete, continuous and mixed
variables. Their only disadvantage is the potentially
large amount of time required to converge to a near-
optimal solution.
Substantial improvements have been made in
increasing the convergence speed of GAs without loss
of the methods' robustness. The use of advanced gen-
etic operators [41] reduces the processor time to a
highly competitive level. Further study [63] shows that
GAs may achieve high levels of parallelism and, with
the use of parallel hardware, ``superlinear'' speedups
may be obtained for engineering problems.

0045-7949/99/$ - see front matter # 1999 Civil-Comp Ltd and Elsevier Science Ltd. All rights reserved.
PII: S 0 0 4 5 - 7 9 4 9 ( 9 8 ) 0 0 2 5 5 - 7
546 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
Attempts to speed up the simulated annealing (SA)
either by performing changes to the algorithm or by
the use of parallel hardware has normally resulted in
problem dependent behaviour. The parallelization of
the SA is not a trivial task because of the sequential
nature of the algorithm. In addition, determining an
ecient mechanism to generate states and selecting an
e€ective annealing schedule require a certain amount
of art.
It is unlikely that parallel SA implementations will
even achieve maximal eciency of the parallel hard-
ware, as frequently happens with GAs. Nevertheless,
as the application of GAs and SAs does not necessarily
overlap their applications, parallel SAs are an ecient
alternative solution procedure for some engineering
problems.
2. Simulated annealing versus genetic algorithm
GAs are population-based instead of point-based as
SAs, i.e. they attempt to evolve a number of complex
systems concurrently while SAs develop one and re®ne
it. The implicit parallel properties [7,26] gained by
evolving a population of points in the search space
concurrently suggests that GAs have a natural map-
ping onto parallel architectures. Revised advanced op-
erators [41] which improve substantially the
performance of the GAs have demonstrated that they
are a powerful multi-purpose optimization tool.
By contrast, SAs can be easily implemented with
quite a minimal degree of coding as compared with
GAs and other nonlinear optimization algorithms.
This feature has made the SA very attractive for hy-
bridizations and also for use in cascade procedures
after the use of quick but less accurate optimizers.
For well-understood problems, SAs may easily be
tailored to generate better than random solutions
which may be extremely valuable to speed up the op-
timization process. This tailoring approach is far more
dicult and not recommended for GAs, since they rely
on a good balance between exploration of space and
exploitation of solutions obtained by complex oper-
ators.
In constrained optimization, SAs may take advan-
tage of their generation mechanism to reduce the time
spent in function evaluations by preventing recalcula-
tions of constraints which are not attached to the vari-
able modi®ed. GAs require all constraints to be
evaluated for each new point in the search space.
For engineering problems involving large numbers
of variables, the binary representation of GA popu-
lations may require large amounts of computer mem-
ory above standard hardware capabilities.
Despite of some behavioural di€erences, simulated
annealing and genetic algorithms share more features
with each other than with conventional optimization
techniques. The exchange of knowledge regarding
these two algorithms has helped their mutual develop-
ment. In fact, the SA can be better viewed as a tech-
nique that compliments the GA in the scope of
optimization. Together they may be able to handle all
kinds of engineering problems and with minimal
requirements when compared with conventional tech-
niques which often require sensitivities and other infor-
mation.
3. Canonical simulated annealing
The common deterministic optimization process fails
when the design variables fall into a valley where a
local optimum exists. Once this happens there is gener-
ally no way out of the valley using mathematical pro-
gramming techniques. Thus, the problem is usually
retried with a new starting point in the hope of ®nding
a global optimum. Since SA accepts an increase of the
objective function value within a certain probability, it
is possible to escape out of the valley by uphill move-
ment and to thus continue the search in a good direc-
tion. Eventually, the SA process will have a good
chance of reaching a global optimum.
The basic concept of SA arises from an analogy of
metallurgical annealing. Annealing denotes a physical
process in which a solid in a heat bath is melted at
high temperatures until all molecules of the melted
solid can move freely with respect to one another, fol-
lowed by cooling until thermal mobility is lost.
The perfect con®guration for a crystal is the one in
which all the atoms are aligned in a low level lattice.
As the metal cools, atoms line up with their neigh-
bours, but in di€erent regions atoms may point in
di€erent directions. Whole regions of atoms must then
be reversed in order to escape this state of local opti-
mum having two unaligned crystals, as shown in
Fig. 1. The energy to do this is available as the heat in
the metal. At any temperature above zero atoms
vibrate with an energy that depends on the tempera-
ture of the metal [54]. This energy may cause atoms to
reorientate themselves with a probability that, in the
canonical simulated annealing (CSA), depends on the
current temperature of the system, given by the
Boltzmann distribution:
Pr…E †0e…ÿE=kB T † …1†
This distribution expresses the concept that a system in
thermal equilibrium at temperature T has its energy
probabilistically distributed among all di€erent energy
states E.
As the temperature goes down it becomes more
dicult for the system to undergo large changes.
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
Fig. 1. Simulated annealing: crystals forming in a cooling
metal.
When the temperature approaches zero, ¯ipping
becomes impossible and the state of the atoms is
frozen (Fig. 2).
In this way, if the cooling is carried out suciently
slowly, the atoms are often able to arrange themselves
in a low energy state and form a pure crystal that is
completely ordered. However, if the cooling is done
too quickly for the solid to reach thermal equilibrium
at each temperature value, the solid forms a polycrys-
talline or metastable amorphous state rather than a
low energy lattice. In this case reheating and cooling is
required to form a pure crystal.
In other words, the system sometimes goes uphill as
well as downhill; but the lower the temperature the less
likely is the possibility of any signi®cant uphill excur-
sion. The algorithm may be formulated by commen-
cing at an initial solution or state that is repeatedly
improved by making small local random alterations to
yield a better solution. A pseudo-C procedure is pre-
sented as follows:
T=T0;
while (T>Tfreezing){
do until (Equilibrium is reached){
Alternation (State i 4 State j);
if (DEij < 0) then
accept update (State j);
else
r=random number [0, 1);
if (exp(ÿDEij/T )>r) then
accept update (State j);
else
refuse update (State i);}
Tn=Tn  Tf ;}
where exp(ÿDEij/T ), known as a Boltzmann factor, is
the probability of acceptance of a deterioration (i.e. a
worse state), and T0=initial temperature,
Tn=temperature at the time n, Ei=the energy value
for the current state, Ej=the energy value for a candi-
date state, Tf is the reduction factor for the tempera-
ture which is a function of the time n and given by:
547
Fig. 2. The Boltzmann probability as a function of the tem-
perature.
1
Tf ˆ …2†
log …1 ‡ n†
4. Convergence speed versus accuracy
Theoretically speaking, when Tn is slowly reduced
by a strict cooling schedule with Tn 4 0 and n 4 1,
the ®nal state will be the global optimal solution with
probability 1. In practice, it is dicult to determine the
length of the Metropolis [50] sampling process at each
value of Tn and how slowly Tn should decrease. The
theoretical formula (2) for which the ergodic property
of the canonical simulated annealing has been proved
is far too slow to be used in practice. Instead, some
fast cooling schedules are usually used for which
reason the ®nal solution is probably worse than some
solutions earlier encountered. This also explains the
fact that sometimes the solution obtained by the SA is
even worse than the solutions obtained by some heuris-
tic methods. Nevertheless, improvements in the gener-
ation mechanism and/or the cooling schedule may
result in more ecient implementations without de-
terioration of the quality of the ®nal solution. An
interesting approach proposed by Szu and Hartley
[59,60] is known as fast simulated annealing (FSA).
FSA uses a Cauchy probability density to generate a
random process which allows large steps more often
than the Gaussian probability used by the CSA. This
approach leads to an inverse linear cooling rate rather
than an inverse logarithmic cooling rate.
A second methodical improvement in SA was pro-
posed by Greene and Supowit [24]. These authors pre-
sented a rejectionless mechanism to generate new
solutions with a probability proportional to the e€ect
of the transition on the cost function. Thus a sub-
sequent solution is chosen from the neighbourhood of
a current solution and no rejection occurs.
548 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564

Other attempts to speed up the SA simply replaced the late 1980 s. In 1988, Elperin [18] described the
expression (2) in order to obtain faster cooling sche- basic ideas of Monte Carlo annealing algorithms for
dules. The exponential cooling schedules (3), or simu- structural optimization using discrete design par-
lated quenching (SQ), as they are known, are the most ameters. In the same year, Salama et al. [56] presented
common in practice. an application of SA to optimal placement of actua-
tors and/or sensors for large ¯exible space structures.
Tn ˆ T0 :an …3† The shape control of large ¯exible space structures is
of great interest to structural designers, because inac-
Some authors [30] claim that their SA-like algorithms
curacies in the length of members and the dimensions
justify an exponential cooling when a particular distri-
of joints of such structures may produce unacceptable
bution is used for generating trials. Catoni [13] pro-
levels of surface distortion and internal forces. After
posed a suitably adjusted cooling schedule which is a
Salama's work, the SA has been employed in at least
function of a ®nite amount N of available computing
three other studies in shape control or accuracy by op-
time where an is given by:
timal actuators in large truss-type structures. Each of
1 them adopted di€erent optimization criteria and
an ˆ …c log N † 2 …4† annealing approaches. In order to accelerate the over-
all convergence, Chen et al. (1991) [14] used the best
However, in general, a is arbitrarily speci®ed with
solution for a starting point every time that the tem-
commonly used values between 0.95 and 0.99 as rec-
perature was reduced. The results showed that the SA
ommended in Laarhoven and Aarts [37]. There is yet
provides a computationally ecient tool to ®nd near
the possibility of making dynamic selections of Tn
optimal solutions for otherwise computationally prohi-
according to the progressively discovered features of
bitive problems. In addition, it was reported that in
the energy landscape. These adaptive cooling schedules
multiple SA runs a number of near optimal solutions
have been presented in a number of studies [37,38]
of the same quality, but corresponding to alternative
with appropriate heuristics.
designs, were identi®ed. Onada [53] used the improved
Kiselyov et al. [35] proposed a SA-like algorithm
simulated annealing (ISA) [36] which is based on the
which attempts to improve the quality of the solution
GA. The ISA showed higher performance when com-
rather than reduce the annealing time. This algorithm
pared with conventional methods, such as worst-out
di€ers from the traditional ones in that ®rst a random
best-in (WOBI) and exhaustive single point substi-
¯uctuation of the feasible costs of the objective func-
tution (ESPS). In the third study, Hamernik [28]
tion is calculated and then a few random rearrange-
employed an adaptive cooling rate which is a function
ments in the state are executed. Hence, only those
of the energy of the current state and the energy and
rearrangements for which the cost does not exceed this
temperature associated with the last accepted state.
random ¯uctuation are accepted.
The results showed that, although SA selects subopti-
Most of these authors claim that their cooling sche-
mal locations, it yields very near optimal solutions at a
dules are more ecient than the existing ones. On the
fraction of the numerical cost of a fully combinatorial
other hand, some authors [2] arm that cooling sche-
search.
dules, in general, cannot substantially improve the al-
Procedures other than active methods have been
gorithm's eciency. Whether these faster or
proposed to reduce the problem of shape inaccuracies
``improved'' algorithms may accomplish their promise
using SA. Kincaid [32] formulated a discrete optimiz-
is questionable since they have only been tested on a
ation problem to minimize distortion in trusses and
very small sets of problems. Nevertheless, the methods
compared SA with tabu search [22,23]. The same
reported in this section present very attractive features
author [33] used the SA in single and biobjective mini-
for parallelization.
mization of surface distortion and internal forces to
Finally, more e€ective improvements in the conver-
reduce the need for active controls. The minimal sol-
gence speed or in the accuracy of the solution of the
utions obtained in many SA runs are, on average,
SA rely either on a priori knowledge of the behaviour
64.5% better than the ones obtained by the best of
of the energy landscape or on combinations with other
two other heuristic methods for member exchange [25]
methods. However, these modi®cations are strongly
in terms of the value of the objective function.
problem dependent.
However, the processor time used in the SA runs were
on average about 38 times slower than the other heur-
istic methods.
5. Simulated annealing in structural optimization Another problem area where SA has found large ap-
plicability is in the optimum design of skeleton struc-
Reports of attempts to use the simulated annealing tures. Balling [4] applied SA to the discrete
algorithm in structural engineering ®rst appeared in optimization of three-dimensional steel frames. Balling
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
used a suitably adjusted cooling schedule and the nor-
malization Boltzmann's constant k as the running aver-
age of E, for which the designs are feasible. The SA
was compared with the linearized branch-and-bound
(LB&B) strategy. The SA proved to be more reliable
and was able to obtain lighter structures than the
LB&B algorithm. The LB&B algorithm is very sensi-
tive to the neighbourhood size. However, with the suit-
able neighbourhood size it converged nearly ®ve times
faster than the SA. Balling also investigated the possi-
bility of using approximate structural analysis in order
to reduce the large time spent in function evaluations.
Although the SA was able to obtain lighter solutions
in a reduced time, in this case the exact analysis
proved these solutions to be infeasible.
Lee and Lee [40] applied SA to minimum weight de-
sign of trusses with discrete variables. Hwa and Soon
(1992) used the same path described in Ref. [4] and
compared the SA with two traditional deterministic
methods: the gradient projection method (GPM) [29]
and the simplex method [52]. Again the SA proved to
be more reliable in locating near optimal solutions, but
slower than the two other methods. In addition, an
approach using design sensitive analysis for all design
increments and employing exact analysis to reduce the
error accumulated by the approximate analysis was
proposed. The same quality of solution when using
exact analysis was obtained in about 40.0% of the
time spent when using approximate analysis. However,
the truss structure used in the numerical example was
too small to permit sound general conclusions. More
conclusive results were obtained by May and Balling
[49] performing parameter sensitive analysis of three-
dimensional steel frames using gradient information to
prevent the analysis of potentially poor designs. This
®ltered SA achieved substantial improvement in the
convergence speed with only slight loss of robustness.
Later, Siarry et al. [57] showed that by using gradient
search direction as a local optimization step for the
SA, it is assured that the solution found is the best in
its neighbourhood.
Topping et al. [62] developed an adaptive search
strategy for a bi-criteria optimization of planar truss
structures with discrete and continuous variables using
the SA method. A number of heuristics were intro-
duced in the generation mechanism to perform changes
in the topology while the cross-sectional areas were
minimized. This generation mechanism was designed
to enable the algorithm to remove the redundant mem-
bers while preventing mechanisms being formed in the
structural behaviour. For small structures where the
potential topological solutions are few, the SA may be
computationally expensive if compared with traditional
methods. However, for structures with a large number
of topological possibilities, an exhaustive search could
be computationally impracticable. In such cases the SA
549
proved to be a viable solution with some attractive fea-
tures.
Recently, encouraging results have been obtained by
Bennage and Dhingra [6] which demonstrate the
robustness of the SA, applying the method to single
and multi-objective optimal design of truss structures
with discrete-continuous variables. To model the mul-
tiple objective functions in the problem formulation, a
cooperative game theory approach was used. To
obtain such results a cooling schedule similar to that
reported in Ref. [4] was used. The di€erence in this
case is that, instead of a ®xed length for the
Metropolis sampling cycle, an adjustable length which
increases as the temperature is reduced was adopted.
The authors announced promising solutions to reduce
the computational burden for structural problems by
using SA-gradient-based hybrid search schemes.
Lombardi [45] applied the SA to the optimization of
composite plates of 16 ply laminates considering the
buckling load using continuous-discrete design vari-
ables. In this relatively small problem it was observed
that a large part of the design space was explored by
the SA algorithm in order to ®nd the optimum design.
Later, Lombardi et al. [46] employed the same
approach to solve problems with a greater number of
plies (32 and 64). For these problems a much smaller
fraction of the design space had to be investigated in
order to ®nd optimum designsÐan average of 11.0%
for the 32 ply laminate and 0.0003% for the 64 ply
laminate.
The application of SA has been also investigated in
the problem of sparse matrix reduction which has sig-
ni®cant relevance to structural designers, since the
analysis of structural systems is greatly dependent on
the solution of sparse matrix equations, where the
speed depends on the ordering of the unknowns.
Armstrong [3] used an SA to obtain pro®les and wave-
fronts as a benchmark to test nodal resequencing al-
gorithms, once the solutions obtained by the SA
proved to be the minimal obtained. The temperature
decrement in Armstrong's SA was controlled by a vari-
able whose formulation was de®ned by analogy to the
statistical physics quantity known as speci®c heat. This
SA, however, proved to be far too slow for general
use. Lewis [43] used the CSA for pro®le and ®ll re-
duction of sparse matrices. In this case, new minimal
pro®les for the matrices tested by Armstrong have
been obtained, simply by varying the CSA control par-
ameters. The drastic reductions obtained in the pro®les
validates the use of SA in some problems where the
same matrix is used repeatedly with the computational
cost being amortized over a large number solutions of
equations. A more e€ective and innovative approach
towards the general use of SAs to the problem of
matrix reduction was presented by Dubuc [17], who
used a very fast traditional algorithm, the Cuthill±
550 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
MacKee (CM) [15] to generate the initial state for the
SA runs. Special cooling schemes and generation
mechanisms which take advantage of the character-
istics of the problem were also proposed. The CM-SA
in cascade obtained bandwidths smaller than the ones
obtained by all traditional methods and in a€ordable
time. More details about the Dubuc's algorithm are
presented later in this paper.
6. Parallel simulated annealing
As it was explained in the previous section, the main
obstacle in the general application of the SA to struc-
tural engineering optimization is its high consumption
of computational time. Parallel processing appears to
be the only viable way to substantially speed up the
method and thus expand its applicability. For fast tai-
lored SAs, parallel implementations may also reduce
the loss of robustness.
The best parallel scheme is still the object of current
research, since the ``annealing community'' has so far
not achieved a common agreement with regards to a
general approach for the serial SA. However, the per-
formance of parallel SA schemes for structural engin-
eering optimization are likely to be problem dependent
as is the case with the serial ones. In addition, the
choice of parallel schemes largely depends on the
architecture capabilities of the parallel hardware avail-
able. Nevertheless, the exhaustive investigation of poss-
ible designs for tailored parallel SA schemes would
require knowledge of each individual problem and is
not in the scope of the present study. Instead, this
paper investigates general SA models according to
their capabilities and requirements for adaption to
structural engineering problems, so that guidelines can
be produced to help structural designers make a suit-
able choice for their speci®c problems.
However, before discussing di€erent parallel SA
models it is necessary to understand the diculties in
the parallel implementation of the CSA. Traditional
parallelization consists of dividing the work over a
number of processors using distribution of data and/or
distribution of tasks. In optimization it can be
obtained by breaking up the problem and/or the algor-
ithm itself. Parallel SAs using distribution of the pro-
blem need to identify statically or dynamically either a
number of operations or a number of changes in such
a way that they do not interact with the evaluation of
the function. Since in structural optimization problems
most time is spent on evaluations of the functions, this
approach could be very e€ective. However, this source
of parallelization depends exclusively on the problem
and will not be discussed in this paper. Examples of
application of this approach to cell-placement have
been presented by Darema et al. [16] and Casoto et al.
[11].
The ®rst constraint in the distribution of the algor-
ithm is imposed by the sequential nature of the
method. The canonical simulated annealing solves the
optimization problem by successive transitions which
are a combined action resulting in the transformation
of the current solution into a subsequent one. This
action involves two main steps which are sequentially
performed: the generation of a state candidate to the
subsequent solution and the application of the accep-
tance criterion.
Parallel transitions described herein are referred to
those which require global information. The generation
mechanism consists of performing small changes in the
state and it is therefore possible to implement without
substantially modifying the optimization process.
Thereafter, for transitions using multiple state changes,
the accurate application of the acceptance criteria
would require the evaluation of the objective function
considering the combined e€ect of multiple changes.
However, such an approach is unlikely to take full ad-
vantage of the parallel processing capabilities.
Alternatively, concurrent changes may be accounted
for using approximations in the acceptance criterion,
but this will a€ect the behaviour of the algorithm con-
siderably. In addition, a mechanism to correct cumu-
lative errors and a rigorous control to assure
convergence are required. The parallel transitions may
adopt approximate and/or accurate acceptance criteria
accounting for single or multiple changes. Global infor-
mation may be used for the assessment of one solution
to be broadcast to all processors, single transition or
for the assessment of di€erent solutions to be updated
in di€erent processors, multiple transitions.
The parallel architectures that are currently available
are usually classi®ed, according to Flynn [20], in
SIMD (single instruction, multiple data ) and MIMD
(multiple instruction, multiple data ) machines. Both
these architectures may be further subdivided into
shared-memory machines (SMM) and distributed mem-
ory machines (DMM), as suggested by Gurd [27]. For
convenience, this paper will refer to all parallel archi-
tectures using Gurd's classi®cation. In DMM, global
information is always obtained through interprocessor
communication. The frequent utilization of global in-
formation introduces high overheads produced by the
large number of interprocessor communications and
by synchronization barriers. Synchronization is used
here in the sense that parallel events are sequentially
evaluated by a centralized control operation, thus
requiring predetermination of the length of time
required by each event or set of events. The use of syn-
chronization operations indicates that, after accom-
plishing their tasks, some processors will be idle for a
period waiting for the de®nition of new tasks from the
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564 551

central control of the program. The programming pro-

blem, then, is to minimize the number of synchroniza-
tion and communication phases while the architectural
problem is to minimize their durations. From the pro-
gramming point of view, in order to reduce the num-
ber of communications, parallel SA schemes may use
either only serial transitions, only parallel transitions
or alternatively both of them. Serial transitions are the
ones that use only information from one processor to
perform a transition, while parallel transitions use in- Fig. 3. Parallel SA scheme using independent and identical
formation from more than one processor. Serial tran- processing.
sitions are the usual transitions associated with
simulated annealing. Thus, parallel SA schemes may
from the stochastic nature of the method which gener-
have communication after each change (only parallel
ally requires a number of runs to give reliability to the
transitions); communication after a number of changes
results. Thus, the computational time of n serial runs
(serial and parallel transitions); and no communication
may be cut to the time of slowest among the n pro-
during the optimization process (only serial tran-
cessors.
sitions).
SAs, in general, ®nd relatively good solutions in
In addition, parallel schemes using only parallel
quite a short time but spend much time re®ning them.
transitions will, in general, require strong synchroniza-
Using IIP, it is expected that the global search on
tion at every transition. On the contrary, schemes
di€erent processors uses a di€erent path, since they
using only serial transitions may not require synchroni-
start from di€erent positions in the sample (di€erent
zation at all. Thus, the designer only has to decide
random seeds). Therefore, improvements may be made
whether it is more advantageous to use a synchronous
in the convergence speed by using fast SAs which are
or an asynchronous regime, on the schemes which have
based on shorter global optimization periods without
parallel transitions. In an asynchronous regime state
loss of robustness.
transitions are evaluated simultaneously and indepen-
At high temperatures (global optimization) the al-
dently. Units continuously generate state transitions
gorithm has high percentages of accepted moves, i.e.
and accept or reject them on the basis of information
changes in the state, between decrements of the tem-
that is not necessarily up to date. In a synchronous
perature. As the temperature reduces only moves
regime, sets of state transitions are scheduled in succes-
which improve the solution tend to be accepted. Those
sive trials. After each trial, the accepted state transition
percentages gradually drop until stabilization at values
is communicated through the network.
equal to or around zero towards the end of the pro-
Finally, as the length of the communication path is
cess. Thus, it is also possible to reduce the period of
proportional to the diameter of the network of pro-
global optimization in each processor using adaptive
cessors, from the architectural viewpoint, the designer
global controls based on the accepted and/or rejected
has to determine the topology of the network to be
moves from all processors. This approach is particu-
adopted and the maximum number of processors to
larly suitable for SMM and is restricted to a small
use. Such a decision has to take into account the opti-
number of processors. Otherwise, a large number of
mal time coecients between computations and com-
communications would be required to update the
munications.
values of global control parameters.
6.1. SA models with only serial transitions

The simplest way and the only possible way to par-
allelize the SA in a DMM, without requiring processor
communication and synchronization throughout the
optimization process, is by placing copies of a serial
SA on a number of processors (Fig. 3). The only com-
munication between processes is to distribute the initial
control parameters to all nodes and gather the ®nal
results to report to the user. Although this technique,
known as independent and identical processing (IIP),
allows an implementation rigorously faithful to the
serial algorithm, it is unlikely to produce signi®cant
speedup. The motivation behind this model comes
6.2. SA models with only parallel transitions
In an interdependent optimization process, once
di€erent changes are concurrently performed on di€er-
ent processors and the e€ects on the energy state are
locally evaluated, the question is ``when and how to
use global information in order to speed up the pro-
cess?''
This section discusses parallel SA models which use
global information every time the state is changed. In
this case, only parallel transitions will occur. The main
di€erence between this class of parallel SAs lies in the
552 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564

way in which global information is used in order to

apply the acceptance criterion.
The application of the acceptance criterion requires
the energy values for two subsequent states or con-
®gurations. When changes and evaluations are per-
formed independently and locally in di€erent
processors, values of energy of global states are not
available and local states have to be used instead.
However, as in the SA many changes are generally per-
formed and rejected for each transition, parallelism
may be employed to generate a population of states
which is used to exploit good solutions before applying
the acceptance criterion. Global information is
required just to select the best of the local states to be
used by the acceptance criterion and to update the sol-
ution.
Fig. 5. Regime of operation of the processors. (a) Scheme
The implementation of these schemes is based on a
All_On. (b) Scheme Turn_On.
master task which controls the process and the number
of slave tasks which generate new states and evaluate
them (Fig. 4). A series of implementations using this bottleneck in the message ¯ow caused by the need for
approach has been presented by Lee and Stiles [39]. In a task controller.
such implementations the processors may operate in a Fig. 6 shows a similar approach which is known as
permanent regime (All_On ) or they may be turned on a clustering algorithm (CA). In this approach the cur-
as the optimization process proceeds (Turn_On ), as is rent state is updated in all processors every time that
shown in one solution is accepted in any of the processors.
Fig. 5. As the processors communicate after every ``Clustering'' refers to the action of combining two or
move, the Turn_On implementation attempts to reduce more processors to generate a single Metropolis
the number of communications and redundant work Sample (MS) simultaneously. In order to prevent erro-
during the beginning of the process when the percen- neously calculated transitions in SMM, a mechanism is
tage of accepted moves is still high. introduced to communicate to all processors that a
These schemes have quite a simple implementation modi®cation has been accepted and that they must
but have a very low degree of parallelism. They are abort their processes before replacing the current sol-
suitable for SMM or for DMM with very few pro- ution by the accepted one. Despite the fact that the
cessors because these schemes are among the ones CA produce a larger number of communications than
which produce the largest number of communications. the DA during the period of high temperatures, it is
They preserve the convergence properties of the
method by using only accurate evaluations of the
energy function. However, they violate the serial de-
cision sequence of the SAs because they do not con-
sider the combined e€ect of changes. As shown in Fig.
4 the problem, from the architectural viewpoint, is the

Fig. 4. Parallel SA scheme evaluating a population for a Fig. 6. Concurrent serial SAs communicating every accepted
single transition. move.
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
normally more ecient since it does not require strong
synchronization.
Another form of exploiting parallelism is by using
the concurrency technique known as speculative compu-
tation [10]. Along with the SAs using IIP, the SAs
using speculative computation are the only algorithms
which faithfully reproduce serial models in DMM.
Speculative computing is also the only way of perform-
ing parallel transitions without violating the serial de-
cision sequence of the SAs. The concept is to perform
the task before knowing whether it is required or not.
If the work is eventually required, it is already done.
Otherwise, the work is wasted. Parallel SA schemes,
using speculative computation, can be implemented in
tree-like architectural topologies where each node
evaluates states incorporating changes of all previous
nodes. A balanced tree will contain all possible sol-
utions within n sequential changes with n being the
number of levels in the tree (Fig. 7). However, only the
solution corresponding to the correct speculation will
return to the master processor, avoiding in this way
bottleneck problems.
In highly constrained engineering problems the per-
centage of accepted changes in the total attempted is
generally very low. In such a case, unbalanced trees
are likely to be more ecient. The design of optimal
trees will then depend on the expected percentage of
accepted changes from the total attempted in a serial
run. Fig. 8 shows some examples of unbalanced trees
which may be used to obtain more ecient systems.
The ®rst tree explores a probability of only one change
being accepted in ®ve attempted, while the other
assumes a very low acceptance probability during the
whole process. In some parallel hardware, which
allows dynamic con®guration of the topology, di€erent
trees may be used in a single parallel run to match the
demand of the di€erent phases of the process. The
schemes using speculative computation may obtain
Fig. 7. An example tree with 15 processors for speculative
computation.
553
Fig. 8. Examples of unbalanced tree with 15 processors for
speculative computation.
high speedups while preserving all important features
of the serial SA models.
All previously discussed schemes adopt an accurate
acceptance criterion with regards to the theoretical
assumptions for the convergence of the method. In
such schemes, parallel transitions, in spite of searching
in di€erent neighbourhoods, always start from a single
point (single transitions). An alternative approach
would be transitions starting from di€erent points
moving to di€erent points (multiple transitions).
However, in the case, transitions do not have a totally
independent direction, as it happens in the IIP,
since these use global information to update the sol-
utions. In applying the acceptance criterion they
attempt to approximate a value for the energy of a
global state as a function of the energy of local states.
Although this approach has scope for maximum paral-
lelism, it has some inconveniences which may substan-
tially restrict its application to structural engineering
problems.
Engineering problems using approximated assump-
tions are in general heavily constrained and with highly
discontinuous space, which makes it very dicult to
obtain good evaluators. Therefore, histograms and
statistical measures of local evaluations may have
questionable signi®cance. Heavy penalties for con-
straint violations may present a ®tness barrier to the
convergence to a solution from the direction of the in-
feasible region. In functions with many local minima
of similar ®tness, the convergence may not occur
within a€ordable computational time.
Another source of inaccuracy in the acceptance cri-
terion may be encountered in an approach known as
the error algorithm (EA) [37]. The error algorithm may
be described as an entire network of processors locally
generating candidate states and using global states,
which are successively updated with the common mem-
ory, as a current state. The notation ``error'' refers to
the fact that the algorithm allows erroneously calcu-
554 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
lated transitions. There is no con¯ict between pro-
cessors and whenever a move is locally accepted it is
immediately updated in the global memory, regardless
of the moves already computed. In addition, this
approach introduces chaos into the calculations of the
energy states because the global state is the last
updated but not necessarily better than the previous
current local state. Calculations of error tolerance have
to be introduced in the acceptance criterion and mech-
anisms to correct accumulated errors are, in general,
required to allow convergence. The applicability of this
last method in engineering optimization, is therefore
highly questionable.
For DMM, schemes using only parallel transitions
tend to require a larger number of communications
than other schemes. Their ratios between instructions
and communications need to be relatively high in
order to produce signi®cant speedup. However, in
some cases they may still be quite ecient for engin-
eering problems which have function evaluations
requiring a large number of instructions.
6.3. SA models with serial and parallel transitions
In an attempt to reduce communication overheads,
a number of SA schemes have been proposed where a
number of steps using only local information are inter-
calated between two steps using global information,
i.e. di€erent processors concurrently run serial SAs
communicating from time to time to allow parallel
transitions (Fig. 9). This category of parallel SA
schemes is very popular, since they balance a substan-
tial degree of parallelism with satisfactory accuracy in
the solution.
Fig. 9. Concurrent serial SAs communicating from time to time.
The ®rst general applicable approach using this type
of scheme is known as the division algorithm (DA) [37].
This approach uses all available processors running
independently serial SAs until all systems reach equili-
brium at a certain temperature, i.e. during one MS.
Thus, every time that the temperature is reduced, there
is a synchronization among processors and the state
associated with the best solution from all processors is
used to restart another MS.
A good compromise would be starting with the DA
during the regime of high temperatures, when the num-
ber of accepted moves is relatively high, and dynami-
cally switching to the CA for the regime of low
temperatures.
7. Massively parallel simulated annealing
There are few examples of massively parallel im-
plementations in the literature [8,11,21,65], applied
either to placement problems or shape detection. In
both cases the approaches are always based on func-
tional distribution, which is problem dependent, and
on error algorithms the suitability of which is question-
able.
The other schemes proposed until now have been
limited to the implementations on small-scale parallel
machines. Most of the parallel implementations follow
the sequential algorithm strictly or with minor devi-
ation. There are limitations in computing parallel
moves in the length of MS, due to constant overlap-
ping in the sample. The iterative improvement beha-
viour inherited by the SA seems to be unsuitable for
massive implementations. In a network with a very
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564 555

large diameter, the gain in speed due to parallelization

may not be sucient to compensate for the high over-
heads in routing messages and/or synchronization.
Parallelization of the discussed SA schemes in large-
scale parallel machines will be able to make use of
only a small number of the available processors. The
question which arises is how to make more ecient
use of the available parallel hardware. The answer
points towards the multiple sources of parallelism,
combined eciency criteria and possibly the develop-
ment of schemes based on the parallel hardware rather
than on the sequential algorithm.
The use of speculative computation may be an
alternative solution to increase the applicability and
eciency of large-scale machines. Recently, Sohn [58]
presented performance results of a parallel SA using Fig. 10. Multiple speculative trees for large-scale parallel ma-
speculative computation to solve the TSP (travelling chines.
salesman problem). The network consisted of 100 pro-
cessors arranged in an unbalanced tree topology as by means of di€usion. The neighbourhoods may use
shown in Fig. 8a. Sohn reported that good perform- di€erent cooling schedules setting distinct global or
ances were obtained using up to 10 processors. local optimization roles for di€erent neighbourhoods.
Thereafter, the increase in the number of processors As the di€usion mechanism may take a number of
was still able to speed up the optimization process yet transitions to transfer information to the opposite net-
the eciency of the network became very poor. The work ends, the neighbourhoods are allowed to re®ne
performance of this approach is also dependent on the temporarily di€erent solutions or minima. This
initial temperature and the size of the problem, hence approach has potential for powerful search, but it
the higher the initial temperature the lower the e- requires a study of appropriate control parameters
ciency. The reason for the weak performance of the which are expected to be di€erent from sequential al-
large unbalanced trees is that the maximum number of gorithms. It also has limitations in the speedup that
processors that can be e€ectively used in one transition can be expected by increasing the number of pro-
is equal to the number of moves required to perform cessors. However, there is no limitation in the number
such a transition. In addition, the higher the tempera-
ture of the system the smaller the number of trials
between two subsequent accepted moves. The maxi-
mum speedup obtained by Sohn's SA was 20 times on
100 processors.
The current study suggests that further speedup may
be obtained by combining speculative computation
with other sources of parallelism. Fig. 10 shows an
example of implementation that combines two sources
of parallelism. The parallelism presented by the div-
ision or clustering algorithms in generating simul-
taneously an MS and the parallelism of speculative
computation by employing many processors to gener-
ate a single transition. In large DMM, it may be poss-
ible to con®gure the network topology in a number of
independent speculative trees which communicate
between each other to generate simultaneously the
same MS. Each of the speculative trees would then be
used to speed up the transitions.
Alternative approaches may be borrowed from par-
allel GA models [63]. Thus, MS may be generated by
overlapping neighbourhoods of network nodes
(Fig. 11). In this approach processors are only allowed Fig. 11. Parallel SA model with overlapping neighbourhoods
to communicate with their direct neighbours and infor- of ®ve processors transferring information over the network
mation will pass from one network region to another by di€usion.
556 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
of processors, since the increase in the number of pro-
cessor does not represent communication overheads.
8. Parallel hybrid simulated annealing
A number of hybrid schemes using simulated anneal-
ing combined with some local optimization techniques
has been presented in the literature [5,48,55]. These
local optimization techniques are used to re®ne the sol-
utions or to generate better than random solutions.
Although they may be able to speedup the sequential
algorithm, they tend to reduce the scope for paralleli-
zation.
Other hybrid approaches are based on combinations
of the SA with genetic algorithms. Mahfoud and
Goldberg [47] proposed an SA-GA approach which
the authors called parallel recombinative simulated
annealing (PRSA). Later, Lin et al. [44] proposed a
similar approach called the annealing genetic algor-
ithm. Both approaches attempt to introduce the im-
plicit parallelism of the GA's multiple-point search
into an annealing framework. This was obtained by
evolving a GA population under a thermodynamic
control. The GA's roulette wheel selection was
replaced by a Boltzmann-type selection operator or
acceptance criteria within a speci®c cooling process.
The important feature of these algorithms is that they
also inherit the capability to map naturally onto paral-
lel architectures.
A slight di€erence in this approach was introduced
by Brown et al. [9] to solve the quadratic assignment
problem. Brown's hybrid parallel GA uses a simulated
annealing algorithm for each solution attempting to
obtain local improvements in each generation.
9. Synchronization and communication overheads
A majority of the parallel SA schemes consists of
interactive models, i.e. models in which there are top-
down and bottom-up network communications.
In SMM the information exchange between pro-
cessors is performed by means of global memory.
Minimization of communication overheads depends on
the choice made between shared and private memory.
The common memory is rather large, and its access
time is much longer than the processor instruction ex-
ecution time. In addition, overlapping in accessing
memory banks decreases the memory access speed.
Each processor has its own fast cache that should pre-
ferably be used to reduce the number of accesses to
the common memory. SAs make use of the common
memory to store global states to be accessed by all
processors. However, in order to prevent inconsistent
state views among the processors, an expensive syn-
chronous update procedure is required before each
evaluation of state change. The synchronous update
procedure uses some form of timing after which a new
value is determined simultaneously for all processors.
Synchronization is used in the sense of locking pro-
cesses in one or more network nodes until their depen-
dencies are acquired. In SMM, synchronization is
made by an arbiter which controls the execution of all
processors and decides how handed-in information will
be used for updating values. The long waiting time
spent while processes are locked leads to a loss of e-
ciency. The possibilities for relaxing synchronization
depend to a large extent on how much error in the
convergence and in the ®nal solution may be tolerated.
In engineering problems these tolerances are generally
small.
In DMM information is directly transferred from a
processor to another by messages through fast links
when processors are connected to each other, otherwise
a routing mechanism is required to handle the message
passing. The communication by message passing sys-
tems is more time consuming than common memory
access. The problems of communication overheads
depend on the size of the messages, on the diameter of
the network of processors and on the topology selected
for the network interconnection. Synchronization is
forced, in order to allow processors to receive global
information and update current values. Alternatively,
arbiters may also be used, in this case though, to store
handle-in message and to decide when values need to
be updated, allowing in this way that processors carry
out their computations asynchronously without the
introduction of waiting times.
10. Numerical examples
For a great number of structural problems, particu-
larly those using the ®nite element method, the math-
ematical model that has to be solved results in a set of
sparse linear equations. Many algorithms which solve
sparse linear equations use elimination techniques in
combination with a reordering of the coecient matrix
to preserve sparcity. The traditional approach for pro-
blems where the coecient is symmetric or structurally
symmetric is to use symmetric (row and column) per-
mutations to reduce the bandwidth, the pro®le, or the
wavefront of the matrix.
The results obtained using the SA have been com-
pared favourably with all traditional methods in redu-
cing the matrix bandwidth. However, the SA has been
considered far too slow for general use. The ®rst
reason for the poor performance of the SA is because
it uses a larger solution space than traditional
methods. SA reduces matrix bandwidth by randomly
selecting two coecients and exchanging their rows
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
and columns in the matrix. The number of allowable
moves nam, for a square matrix of order N, is given by:
A…A ÿ 1†
nam ˆ , A ˆ …N ÿ 1†…N ÿ 2† …
2 bandwidths obtained by the Cuthill±MacKee algor-
However, a move may only result in a reduced band-
width if it involves one of the noc o€ensive coecients
which are increasing bandwidth. Thus, there is a upper
bound num for the moves which may e€ectively reduce
the matrix bandwidth given by:
num ˆ noc …A ÿ 1† …noc <N † …
For medium to large matrices nam is much bigger than
num and a blind search results in an extensive search in
ine€ective regions. Dubuc [17] proposed a tailored but
very ecient serial SA algorithm for bandwidth re-
duction which uses this a priori knowledge. This
author developed a mechanism to identify possibly
e€ective moves, incorporated in the SA generation
mechanism. This ecient generation mechanism com-
bined with adaptive cooling schedules results in faster
and more e€ective algorithms which are used in this
paper as serial models for parallelization. In the cur-
rent work two modi®cations were performed to the
Dubuc's original serial algorithm in order to make it
consistent with parallel versions. The way in which
random numbers are generated in the original algor-
ithm is machine dependent and had to be modi®ed.
The second modi®cation was the introduction of a
stop criterion in terms of the ratio between number of
accepted moves and the total number of attempted
moves. This last modi®cation was introduced to take
into account the contribution of many processors to
generate a MS in the parallel implementation. The
®nal results in most of the cases were identical to the
ones obtained using the original algorithm.
Everstine's collection [19], consisting of 30 structural
engineering matrices, is used to investigate the per-
formance of the parallel implementation. These
matrices were collected from US military and NASA
users of the structural engineering package
NASTRAN [51] for use as a benchmark test for band-
width reordering algorithms. Table 1 presents some in-
formation about the patterns and minimal bandwidths
of these matrices. In the ®rst two columns in Table 1,
borrowing the jargon from the graph theory, are the
order N of the matrices and the maximum degree, i.e.
the maximum number of connectivities of a generic
node. As the length of an MS and the stop criteria are
functions of the order of the graph corresponding to
the matrix of dimension N  N, the execution time is
highly dependent on the value of N. In addition, for
matrices with high degree of connectivity the band-
widths tend to reduce by very small decrements with a
higher number of accepted moves. The next two col-
557
umns describe the degree of sparcity of the matrices;
the number of entries in the lower triangular part of
the matrices including the diagonal; and the percen-
tages of non-zero coecients in the matrices. The
5†
ithm and by the SA are shown in columns ®ve and six.
It is important to point out that these problems pre-
sent in general a large number of local minima. There
may also be a large number of possible moves which
will not directly a€ect the bandwidth size, although
these moves may have great signi®cance for further
6† bandwidth reductions. These cannot be a priori deter-
mined. The optimization process will wander among
equivalent states which makes it very hard for algor-
ithms to ®nd an appropriate search direction. Many
important intermediary states are lost due to the poor
capabilities of the evaluator which is used to access the
suitability of the solution.
10.1. The serial algorithm
The serial SA model is extensively explained in Ref.
[17] and it is not the intention of this paper to overlap
with the previous work. However, it is important to
explain brie¯y parts of the algorithm that a€ect the
selection of the parallel implementation.
The main aspect to be considered in the choice of
the parallel implementation is the generation mechan-
ism. The improved generation mechanism, proposed
by Dubuc, always uses one of the coecients which is
found at the extremity of the bandwidth for the (row
and column) permutations. Starting from the upper-
left corner, the algorithm moves through the matrix
along the diagonals searching for the ®rst non-zero
coecient ar,c. The current bandwidth is given by
(r ÿ c ). The second coecient for the permutation is
selected at random. Thereafter, a number of heuristics
are used to determine whether the permutation would
increase the bandwidth or not. Thus, if the bandwidth
increases, the new o€ensive is scanned using a bounded
search in the direction of the upper-left corner. The
move is accepted or rejected according to a certain
probability. However, if the bandwidth does not
increase, the move is automatically accepted and the
new non-zero coecient is searched for along the diag-
onals from the position where it has previously
stopped. There may be a number of non-zero coe-
cients which are causing the same maximum band-
width. In order to prevent bias in the choice of one of
these coecients, the diagonals may be considered as
rings, as shown in Fig. 12. Thus, after transversing a
diagonal, the algorithm moves to a random position in
the next diagonal and travels along it forward or back-
ward with equal probability.
This improvement in the generation mechanism, pro-
posed by Dubuc, results in more ecient search and
558 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564

Table 1
EverstineÂs matrix collection: patterns and bandwidths

Bandwidth

Matrix order Maximum degree Symmetric entries Fill (%) Initial CM SA Description

59 5 163 7.67 25 8 6 2D frame

66 5 193 7.35 44 3 3 Truss
72 4 147 4.28 12 7 5 Grillage
87 12 314 7.15 63 18 10 Tower
162 8 672 4.50 156 16 12 Plate with hole
193 29 1843 9.38 62 44 32 Knee prosthesis
198 11 795 3.55 36 12 8 Reinforced mast
209 16 976 3.99 184 33 23 Console
221 11 925 3.34 187 15 13 Hull-tank region
234 9 534 1.52 48 25 11 Tower with platform
245 12 853 2.43 115 40 21 Carriage
307 8 415 0.55 63 37 29 Power supply housing
310 10 1379 2.55 28 13 12 Hull with re®nement
346 18 1786 2.69 318 47 30 Deckhouse
361 8 1657 2.26 50 15 14 Cylinder with cap
419 12 1991 2.03 356 32 26 Barge
492 10 1824 1.30 435 31 17 Piston shaft
503 24 3265 2.38 452 54 41 Baseplate
512 14 2007 1.34 73 32 21 Submarine
592 14 2848 1.46 259 41 29 CVA bent
607 13 2869 1.39 147 52 39 Wankel rotor
758 10 3376 1.04 200 29 20
869 13 4075 0.96 586 41 31
878 9 4163 0.97 519 34 25 Plate with insert
918 12 4151 0.88 839 50 32 Beam with cutouts
992 17 8868 1.70 513 53 37 Mirror
1005 26 4813 0.85 851 102 56 Baseplate
1007 9 4791 0.85 986 34 28
1242 11 5834 0.68 936 92 56 Sea chest
2680 18 13,853 0.35 2499 69 54 Destroyer

substantial speedup but reduces the room for paralle-

Fig. 12. Mechanism to identify an o€ensive coecient in a
sparse matrix.
lism because a pair of coecients can only be gener-
ated after evaluating the e€ect in the bandwidth
caused by the previous permutation.
A second approach was also proposed by Dubuc
[17]. This approach uses the traditional Cuthill±
Mackee, which is one of the fastest algorithms for
bandwidth reduction, to generate initial solutions for
the SA. Thereafter a quasi-quenching cooling schedule
is performed in cascade. Dubuc shows that such an
approach produces, in general, better solutions than
other traditional algorithms in equivalent amounts of
computational time.
In this paper some of the parallel models described
will be investigated for the parallelization of the two
algorithms proposed by Dubuc. The problem of mini-
mization of the bandwidth of sparse matrices was cho-
sen as an example because it exhibits some of the
inconveniences which often make parallelization a di-
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
cult task. In this type of problem the function evalu-
ation (checking the bandwidth) is very fast and the
time between communications is very short.
10.2. The hardware setup
The parallel hardware consists of a 486-PC host and
a number of TRAMs (transputer module) supported
by TTM3 daughterboards. The basic component is an
IMST800 transputer of 25 MHz speed. It is a 32-bit
microprocessor, with 1, 2, 4 and 8 Mbytes of dynamic
RAM, 4 Kbytes of on-chip fast static memory and
FPU (¯oating point unit), delivering 30 MIPS and 4.3
M¯ops peak performance. Communication between
transputers is supported by four bi-directional, serial
links operating at 20 Mbits/s.
The available hardware has been arranged in two
independent distributed memory systems. The ®rst
setup consists of the host 486 PC, a root processor of
2 Mbytes of dynamic RAM and 11 slave processors of
1 Mbyte RAM. The 12 processors were connected in a
pipeline topology using the four available physical
links to handle message passing as shown in Fig. 13.
The two ®rst links are used to connect a router task to
two other adjacent routers in the pipeline. The router
task stores the messages in an internal memory to pro-
tect the messages from being erroneously overwritten
in the worker task. However, the limited amount of
memory does not permit a large volume of data to be
duplicated in internal memories of the router and
Fig. 13. The hardware setup and task distribution in a pipeline topology.
559
worker tasks. On the other hand, if the data are bro-
ken into small pieces to be passed in parts via small
bu€ers, the communication process becomes more time
consuming. Thus, the routers are only responsible for
high priority and short communications between
workers and an arbiter in the master task which is re-
sponsible for selecting the processor that will update
the others. The updates will be communicated directly
to worker tasks through the two other processor links,
from the chosen processor to the others. This ®rst
hardware setup was used to investigate the speedup
due to increases in the number of processors, but the
limited amount of memory does not allow the ex-
ecution for large matrices.
Alternatively, a second hardware system consisting
of a PC host, an 8 Mbytes root processor and three
slaves with 4 Mbytes memory RAM in each, was used
to obtain new minimum bandwidth values for the
large matrices.
10.3. The parallel implementations
A total of seven parallel implementations were used
for the comparisons of the numerical examples. The
®rst ®ve implementations have the modi®ed Dubuc's
SA as the serial model for the parallelization. For the
other two parallel implementations, the SA in cascade
with the Cuthill±MacKee algorithm were used as the
serial model.
The SA's ergodic property and statistical guarantee
560 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
of ®nding a global minimum rely on the assumption of
time t 4 1. In practice, there is a trade-o€ between
the computational time and the quality of the solution.
The direct consequence is that di€erent solutions are
often obtained from independent runs, when the ex-
ecution starts from di€erent points of the sample. In
addition, the runs which are faster in identifying the
valley of a global minimum also tend to be faster to
re®ne the solution. The reliability of the solution is
then based in a few runs which can be performed con-
currently. Thus, the ®rst parallel implementation which
is investigated in this section is an example of IIPSA
as described in Section 6.1. In order to obtain ad-
ditional speedup, the cooling schedule in the IIPSA
adopts a slightly higher freezing point than in the serial
SA, since it is expected that some processors reach the
solution in shorter time than others.
The second parallel implementation used in this sec-
tion consists of slave tasks on all processors evaluating
the e€ect of one move and informing the master task
on the root processor of the results. The master task
will then select one of the moves to be performed by
all slave tasks using two di€erent selection criteria in
alternative implementations. In the ®rst implemen-
tation (PSA1), the state which results in a minimal
bandwidth is selected. In the other implementation
(PSA2) there is a random selection among the accepted
moves, if more than one move is accepted, or a ran-
dom selection among all attempted moves when none
of the moves are accepted. The accepted move is used
to obtain the same current state on all processors and
each of these then evaluates another independent
move. The messages, in this case, are short in length
but are large in number. They consist of the minimal
information which is required to allow the slave tasks
to perform concurrently the same move. The execution
time for these implementations increases with the
increments in the number of processors. The results
start to deteriorate after three processors. This negative
performance of such implementations was expected in
this problem since the time to evaluate a change is
very short and the overheads in communications and
comparisons among solutions are very high. The loss
in the quality of the solution in many problems occurs
because the competition between processors results in
powerful local optimization but reduces the capability
of the algorithm of escaping local minima. The results
obtained by PSA2 were, in general, better than the
ones obtained by PSA1 in terms of quality of the sol-
ution.
An approach similar to that used by the CA
described in Section 6.2, was also implemented for this
small distributed memory system. In this implemen-
tation the accepted move which is ®rst communicated
to the master task is used to update the solutions on
all processors. The counter of accepted moves is incre-
mented on the master task and broadcast to the slave
tasks. This value is then used to decide whether the
subsequent arriving solution has considered the pre-
vious move or not. If the counter value in the message
arriving from a slave task is equal to the one in the
master task, the solution is accepted and communi-
cated to all slaves. However, if the value in the arriving
message is smaller than the one in the master task, it
means that the message was accepted in the slave task
before it has received the latest changes. The message
is then disregarded.
This scheme produced results with much better qual-
ity than the previous two schemes. This algorithm
exhibits slightly di€erent behaviour than the serial SA.
It ®nds the neighbourhood of the global optimum
quicker than the serial algorithm but has more pro-
blems in re®ning them in the last stages. There is often
a loss of one or two units in the accuracy of the ®nal
bandwidth. The frequency of the communications is
gradually reduced towards the end of the process,
since accepted moves become less likely to occur. The
messages in this scheme are practically the same size of
the messages in the two previously implemented
schemes but the number of communications is substan-
tially reduced. This approach results in a speedup for
the ®rst few processors. However, this speedup is
rather due to the new features of the parallel scheme
than to the hardware acceleration. If more processors
are used the frequency of communication increases
and the improvements which may be gained by carry-
ing out the search concurrently on di€erent parts of
the sample are not sucient to compensate for the
overheads in routing massages and bottleneck pro-
blems. However, a limitation of four processors was
observed in order to assure a suitable MS length for
small to medium matrices. Although, for large
matrices, one or two more processors may be used
without loss in the quality of the solution, this is unli-
kely to produce further speedup since there is an
increase in communications overheads due to a bottle-
neck problems. Contrary to what normally occurs in
di€erent serial runs for a same problem, the di€erence
between execution time of parallel runs using di€erent
starting points and di€erent random seeds is minimal.
In addition, the same minimal or near minimal values
were obtained in ®ve independent runs for each pro-
blem in 27 of the 30 problems. The average execution
times of the ®ve serial runs were compared with the
average of other ®ve parallel runs for each problem.
The speedups obtained using this scheme for 29 pro-
blems are shown in Table 2. It was not possible to
measure the speedup for the last matrix due to a limi-
tation of memory. The minimal bandwidths for the
last matrix were obtained in simulations of the parallel
schemes implemented in a SUN/Sparc10 workstation.
The next parallel SA scheme which was implemented
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564 561

Table 2
Minimal bandwidth and speedup of parallel implementations

Speedup

Clustering A Division A

Number of Processors

Matrix order Minimum bandwidths 2 3 4 2 3 4

59 6 1.216 1.233 1.242 ÿ 1.101 ÿ 1.134 ÿ 1.182

66 3 1.182 1.319 1.412 1.131 ÿ 1.112 ÿ 1.143
72 5 1.372 1.491 1.648 ÿ 1.119 ÿ 1.162 ÿ 1.195
87 10 1.100 1.197 1.208 ÿ 1.081 ÿ 1.097 ÿ 1.122
162 11a 1.047 1.065 1.115 ÿ 1.032 ÿ 1.037 ÿ 1.070
193 32 1.017 1.016 1.085 1.017 ÿ 1.028 ÿ 1.046
198 8 1.048 1.197 1.201 ÿ 1.008 ÿ 1.089 ÿ 1.172
209 23 1.038 1.118 1.209 1.045 1.046 1.037
221 13 1.099 1.201 1.244 1.058 1.073 1.066
234 11 1.143 1.189 1.241 1.091 1.112 1.102
245 21 1.122 1.212 1.306 1.067 1.099 1.126
307 28a 1.092 1.167 1.192 1.045 1.113 1.140
310 12 1.123 1.195 1.271 1.094 1.141 1.167
346 29a 1.100 1.178 1.219 1.033 1.078 1.107
361 14 1.088 1.153 1.195 1.103 1.138 1.155
419 25a 1.163 1.225 1.290 1.056 1.097 1.134
492 17 1.091 1.156 1.198 1.111 1.172 1.181
503 41 1.113 1.167 1.200 1.135 1.162 1.173
512 21 1.177 1.256 1.303 1.094 1.146 1.187
592 29 1.114 1.162 1.193 1.132 1.159 1.180
607 38a 1.083 1.142 1.190 1.101 1.173 1.174
758 20 1.107 1.176 1.223 1.082 1.131 1.163
869 31 1.112 1.166 1.202 1.113 1.152 1.189
878 24a 1.151 1.203 1.254 1.092 1.164 1.128
918 32 1.108 1.139 1.182 1.130 1.179 1.211
992 35a 1.063 1.093 1.145 1.165 1.193 1.214
1005 28 1.107 1.176 1.236 1.103 1.144 1.167
1007 37 1.123 1.182 1.214 1.110 1.162 1.173
1242 56 1.137 1.177 1.201 1.101 1.148 1.181
2680 52a ± ± ± ± ± ±
a
New minimal bandwidths obtained in runs using the DA scheme.

is an example of the DA, described in Section 6.3, for
DMM. In this implementation each processor runs an
entire and independent MS between two communi-
cations. The time between communication, i.e. the dur-
ation of one MS, is de®ned either by a speci®ed
number of accepted moves or by a upper bound value
of attempted moves, whichever is achieved ®rst. After
a slave task has communicated its results to the master
task, it waits for the decision of the master task con-
cerning the processor with the best move. This pro-
cessor then broadcasts throughout the network its
values, which are used as starting parameters for a
subsequent MS. For sparse matrix problems the time
to reverse the accepted moves in the other processors
and repeat the same sequence of accepted moves which
was performed by the processor with the best move
would be very long. Instead, the entire state array and
all relevant data from this processor are passed to the
other processors. Thus, this scheme also presents a
reduced number of communications when compared
with the algorithms PSA1 and PSA2. However, the
size of the messages may be much larger, since in this
case the messages contain the entire set of values corre-
sponding to the current state. This scheme also showed
some speedup, but only for the matrices of medium to
high order. For small matrices, the frequency of com-
munication is too high to allow any speedup. Even for
the large matrices, this scheme was still slightly slower
than the previous one but the quality of the results in
general are equal or better than the serial one. The
562 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
limitation to a few processors remains. In addition, the
more processors are used the shorter the time between
communications. The results obtained using this
scheme are shown in Table 2.
The two last approaches were used to parallelize the
SA part of the Dubuc's CM-SA running in cascade. In
this case, the cooling starts from a much lower tem-
perature than the conventional SA. The accepted
moves become less frequent and the performance of
the parallel schemes improves in terms of speed. The
quality of solution also improves when compared with
the serial CM-SA using equivalent running times. For
the majority of the cases the minimum solution was
obtained in computational time compared with the tra-
ditional methods which are used in practice.
11. Concluding remarks
The problem of bandwidth reduction used in the nu-
merical examples shows some critical diculties for the
parallelization of the SA. The time which is spent to
generate and evaluate a new solution is in general
shorter than the time to communicate a solution to
master task, to deliberate about the solution and com-
municate the decision back to the network nodes. The
size of the messages may be relatively large. In ad-
dition, the deterministic generation mechanism of the
selected serial model reduces the degree of parallelism
and the options for parallel implementations.
However, for the great majority of the engineering pro-
blems, function evaluations are likely to be very com-
putationally expensive when compared with the time
for passing messages. In such a case, designers may
have a wider range of choices for ecient paralleliza-
tion of the SA.
The IIPSA may not represent the most ecient util-
ization of the parallel hardware. Although some
speedup may be obtained by using faster cooling sche-
dules, these are in general not substantial enough to
justify loss of the quality of the solution. This class of
algorithms may be of some applicability for problems
where the accuracy is the determining factor in the
selection of the parallelization scheme and when it is
important to preserve the characteristics of the serial
model. In the IIPSA the savings in the time are not
sucient to make it ecient for minimizing the band-
width of sparse matrix.
Parallel implementations with communications after
every move, such as PSA1 and PSA2 may only be
e€ective when the function evaluation is computation-
ally expensive. They are suitable for convex problems
or problems with very few minima, since they perform
intensive local search but a poor global search.
The DA and CA are the most attractive approaches
for engineering problems, in terms of both convergence
speed and quality of the solution. Depending on the
selected cooling schedule these algorithms may present
a behaviour di€erent from the serial model. In some
applications, such a behavioural transformation may
enhance the algorithm's capabilities. For engineering
problems these schemes have potential for ``super-
linear'' speedup because the time to evaluate a function
is in general much longer than the time to broadcast a
solution. There is, however, a limitation in the number
of processors which is in general determined by a mini-
mum length of sampling required for an MS. If the
MS period becomes very short, the quality of the sol-
ution deteriorates. Additional speedup may possibly be
obtained by the use of speculative computation.
Parallelization appears to be the only general
approach to improve the performance of the SA and
expand its applicability to engineering optimization.
11. Unlinked References
ref [1, 12, 31, 34, 42, 61, 64]
Acknowledgements
The authors acknowledge, with grateful thanks, the
work of Eduardo Dubuc who developed the serial SA
models employed here and kindly lent them for the
development of the current study. The ®nancial sup-
port from Brazilian Ministry of Education, CNPq, is
also gratefully acknowledged.
References
[1] Aarts EHL, De Bont FMJ, Habers JHA, Korst JHM.
Parallel implementations of the stochastical cooling al-
gorithm. Integration 1986;4:209±38.
[2] Aarts EHL, Korst JHM. Simulated annealing in
Boltzmann machines: a stochastic approach to combina-
torial optimization and neural computing. New York:
Wiley, 1989.
[3] Armstrong BA. Near-minimal matrix pro®les and wave-
fronts for testing nodal resequencing algorithms.
International Journal for Numerical Methods in
Engineering 1985;21:1785±90.
[4] Balling RJ. Optimal steel frame design by simulated
annealing. Journal for Structural Engineering
1991;117:1780±95.
[5] Baluja S. Local optimization using simulated annealing.
In: IEEE Proceedings of the International Conference on
Systems, Man and Cybernetics, 1992, 1, p. 583±588.
[6] Bennage WA, Dhingra AK. Single and multiobjective
structural optimization in discrete-continuous variables
using simulated annealing. International Journal for
Numerical Methods in Engineering 1995;38:2753±73.
 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
[7] Bertoni A, Dorigo M. Implicit parallelism in genetic al-
gorithms. Arti®cial Intelligence 1993;61:307±14.
[8] Bongiovanni G, Crescenzi P. A parallel simulated
annealing for shape detection. Computer Vision and
Image Understanding 1995;61:60±9.
[9] Brown DE, Huntley CL, Spilane AR. A parallel genetic
heuristic for the quadratic assignment problem. In:
Proceedings of the Third International Conference on
Genetic Algorithms, 1989. p. 406±15.
[10] Burton FW. Speculative computation, parallelism and
functional programming. IEEE Transactions on
Computers 1985;34:1190±3.
[11] Casoto A, Romeo F, Sangiovanni-Vincentelli AN.
Placement of standard macro-cells using simulated
annealing on the connection machine. In: IEEE
International Conference on Computer-Aided Design,
Santa Clara, 1987. p. 350±3.
[12] Casoto A, Romeo F, Sangiovanni-Vincentelli AN. A
parallel simulated annealing for placement of macro-
cells. IEEE Transactions on Computer-Aided Design
1987;6:838±47.
[13] Catoni O. Rates of convergence for sequential annealing:
a large deviation approach. In: Azencott R, editor.
Simulated annealing: parallelization techniques. New
York: Wiley, 1992. p. 25±36.
[14] Chen G-S, Bruno RJ, Salama M. Optimal placement of
active/passive members in truss structures using simu-
lated annealing. AIAA Journal 1991;29:1327±34.
[15] Cuthill E, McKee JM. Reducing the bandwidth of sparse
symetric matrices. In: Proceedings of the 24th National
Conference ACM, New York, 1969. p. 157±72 ACM
Publication P69.
[16] Darema F, Kirkpatrick S, Norton VA. Parallel algor-
ithms for chip placement by simulated annealing. IBM
Journal of Research and Development 1987;31:391±402.
[17] Dubuc EJ. Bandwidth reduction by simulated annealing.
International Journal for Numerical Methods in
Engineering 1994;37:3977±98.
[18] Elperin T. Monte Carlo structural optimization in dis-
crete variables with annealing algorithm. International
Journal for Numerical Methods in Engineering
1988;26:815±21.
[19] Everstine GC. A comparison of three resequencing algor-
ithms for the reduction of matrix pro®le and wavefront.
International Journal for Numerical Methods in
Engineering 1979;14:837±53.
[20] Flynn MJ. Very high speed computer systems. IEEE
Proceedings 1966;54.
[21] Gaudron I, Trouve A. Massive parallelization of simu-
lated annealing: an experimental and theoretical
approach for spin glass models. In: Azencott R, editor.
Simulated annealing: parallelization techniques. New
York: Wiley, 1992. p. 163±86.
[22] Glover F. Tabu search, part I. ORSA Journal of
Computing 1989;1:190±206.
[23] Glover F. Tabu search, part II. ORSA Journal of
Computing 1990;2:4±32.
[24] Greene JW, Supowit KJ. Simulated annealing without
rejected moves. IEEE Transactions on Computer-Aided
Design 1986;5:221±8.
[25] Greene WH, Haftka RT. Reducing distortion and in-
563
ternal forces in truss structures by member exchanges.
NASA Technical Memorandum 101535, 1989.
[26] Grefenstette JJ, Baker JE. How genetic algorithms work:
a critical look at implicit parallelism. In: Proceedings of
the Third International Conference on Genetic
Algorithms, 1989. p. 70±9.
[27] Gurd JR. A taxonomy of parallel computer architec-
tures. In: Proceedings of the International Conference on
the Design and Applications of Parallel Digital
Processors. Lisbon, Portugal: IEE, 1988.
[28] Hamernik TA. Optimal placement of damped struts
using simulated annealing. Journal of Spacecraft and
Rockets 1995;32.
[29] Haug EJ, Arora JS. Applied optimal design. New York:
Wiley, 1979.
[30] Ingber L. Very fast simulated re-annealing. Mathematical
Computer Modelling 1989;12:967±73.
[31] Kim GH, Yang YS. Stochastic search techniques for glo-
bal optimization of structures. In: Proceedings of Korea±
Japan Joint Seminar on Structural Optimization. Seoul,
Korea, May, 1992. p. 87±95.
[32] Kincaid RK. Minimizing distortion in truss structures: a
comparison of simulated annealing and tabu search.
AIAA Paper, No. 91-1095, p. 424±430, 1991.
[33] Kincaid RK. Minimizing distortion and internal forces in
truss structures via simulated annealing. Structural
Optimization 1992;4:55±61.
[34] Kirkpatrick S, Gelatt Jr CD, Vecchi MP. Optimization
by simulated annealing. Science 1983;220:671±80.
[35] Kiselyov BS, Kulakov N Yu, Mikaelian AL.
Modi®cation of the simulated algorithm method for sol-
ving combinatorial optimization problems. In:
Proceedings SPIEÐPhotonics for Computers, Neural
Networks and Memories. July, 1992. 1773, p. 120±124.
[36] Koakutsu S, Sugai Y, Hirata H. Block placement by
improved simulated algorithm based on genetic algor-
ithm. In: Proceedings of the Fifth Conference on Systems
Modeling and Optimization. Zurich: IFIP, 1991. 180, p.
648±656.
[37] Laarhoven van PJM, Aarts EHL. Simulated annealing:
theory and applications. Dordrecht: Reidel, 1987.
[38] Lam J, Delosme J-M. Performance of a new annealing
schedule. In: Proceedings of the 25th ACM/IEEE Design
Automation Conference, 1988. p. 306±11.
[39] Lee FH, Stiles GS. Parallel simulated annealing: several
approaches. In: Board Jr JA, editor. NATUG 2, transpu-
ter research and applications, vol. 2, 19??.
[40] Lee C-D, Lee WD. Optimal truss design by stochastic
simulated annealing. In: Proceedings of the Korea±Japan
Joint Seminar on Structural Engineering. Seoul, 1992. p.
191±200.
[41] Leite JPB, Topping BHV. Parallel genetic models for en-
gineering optimization. In: Proceedings of the Structural
Engineering Computational Technology Seminar.
Edinburgh: Civil-Comp Press, 1995. p. 79±95.
[42] Leite JPB, Topping BHV. Genetic algorithm based opti-
mizers for engineering optimization. In: CIVIL-
COMP95, The Sixth International Conference on Civil
and Structural Engineering Computing: Developments in
Computational Techniques for Structural Engineering.
Edinburgh: Civil-Comp Press, 1995. p. 151±65.
564 J.P.B. Leite, B.H.V. Topping / Computers and Structures 73 (1999) 545±564
[43] Lewis RR. Simulated annealing for pro®le and ®ll re-
duction of sparse matrices. International Journal for
Numerical Methods in Engineering 1994;37:905±25.
[44] Lin F-T, Kao C-Y, Hsu C-C. Applying the genetic
approach to simulated annealing in solving some NP-
hard problems. IEEE Transactions on Systems, Man and
Cybernetics 1993;23:1752±67.
[45] Lombardi M. Ottimizzazione di lastre in materiale com-
posito con l'uso di un metodo di annealing simulato,
Tesi di Laurea, Department of Structural Mechanics,
University of Pavia, Italy 1990.
[46] Lombardi M, Haftka RT, Cinquini C. Optimization of
composite plates for buckling by simulated annealing.
AIAA paper, No. 92-2313-CP, 1992.
[47] Mahfoud SW, Goldberg DE. Parallel recombinative
simulated annealing: a genetic algorithm. IlliGAL Report
No. 92002, Department of General Engineering,
University of Illinois, Urbana, IL, 1992.
[48] Martin OC, Otto SC. Combining simulated annealing
with local search heuristics. In: Laporte G., Osman I.,
editors. Metaheuristics in combinatoric optimization,
1993 (submitted).
[49] May SA, Balling RJ. A ®ltered simulated annealing strat-
egy for discrete optimization of 3D steel frameworks.
Structural Optimization 1992;4:142±8.
[50] Metropolis N, Rosenbluth A, Teller A, Teller E.
Equation of state calculations by fast computing ma-
chines. Journal of Chemical Physics 1953;21:1087±92.
[51] NASA. The NASTRAN theoretical manual.
Washington, DC: NASA SP-221 (03), 1976.
[52] Nelder JA, Mead A. A simplex method for function
minimization. Computer Journal 1965;7.
[53] Onoda J. Actuator placement optimization by genetic
and improved simulated annealing algorithms. AIAA
Journal 1993;31:1167±9.
[54] Radcli€e N, Wilson G. Natural solutions give their best.
New Scientist 1990;1712.
[55] Sakurai T, Lin B, Newton R. Fast simulated di€usion:
an optimization algorithm for multiminimum problems
and its application to MOSFET model parameter extrac-
tion. IEEE Transactions on Computer-Aided Design
1992;11:228±33.
[56] Salama M, Bruno R, Chen G-S, Garba J. Optimal place-
ment of excitations and sensors by simulated annealing.
In: Recent advances in multidisciplinary analysis and op-
timization. NASA CP-3031, 1990. p. 1441±58.
[57] Siarry P, Dupinet E, Mekhilef M. A new way to opti-
mize mechanical systems using simulated annealing. In:
Hernandez S, Brebia CA, editors. Optimization of struc-
tural systems and applications. Southampton:
Computational Mechanics Publications, 1993. p. 569±83.
[58] Sohn A. Parallel N-ary speculative computation of simu-
lated annealing. IEEE Transactions on Parallel and
Distributed Systems 1995;6:997±1005.
[59] Szu H, Hartley R. Fast simulated annealing. Physics
Letters A 1987;122:157±62.
[60] Szu H, Hartley R. Nonconvex optimization by fast simu-
lated annealing. Proceedings of the IEEE 1987;75:1538±
40.
[61] Thornton AC. Genetic algorithms versus simulated
annealing: satisfaction of large sets of algebraic mechan-
ical design constraits. In: Gero JS, Sudweeks F, editors.
Arti®cial intelligence in design '94. Dordrecht: Kluwer
Academic, 1994. p. 381±98.
[62] Topping BHV, Khan AI, Leite JPB. Topological design
of truss structures using simulated annealing. In: CIVIL-
COMP93, The Fifth International Conference on Civil
and Structural Engineering Computing and Arti®cial
Intelligence, Neural Networks and Combinatorial
Optimization in Civil and Structural Engineering.
Edinburgh: Civil-Comp Press, 1993. p. 151±65.
[63] Leite JPB, Topping BHV. Parallel genetic models for en-
gineering optimization. In: Proceedings of the Structural
Engineerimg Computational Technology Seminar.
Edinburgh: Civil-Comp Press, 1995. p. 79±95 May.
[64] Witte EE, Chamberlain RD, Franklin MA. Parallel
simulated annealing using speculative computation. IEEE
Transactions on Parallel and Distributed Systems
1991;2:483±94.
[65] Wong C-P, Fiebrich R-D. Simulated annealing-based cir-
cuit placement algorithm on the connection machine sys-
tem. In: Proceedings of the International Conference on
Computer Design, 1987. p. 78±82.