As already stated, the synaptic weights are initialized to the yield those output vectors.
those output vectors. The present network only classifies
normalized input vector matrix as
[W] = [i]
When a new mode shape with noise, as shown in Fig. 4, is
presented as
(X) = (0.1, 0.5, 0.35, 1.0)
the corresponding normalized input is
(l) = (0.08, 0.42, 0.29, 0.85)
The output vector is calculated as
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 10/23/18. Copyright ASCE. For personal use only; all rights reserved.
(l). [i] = (0.92, -0.142,0.285, -0.175)
which shows mode 1 is the winner and represents the first
mode shape; similarly, other classes can be determined. Hence
the procedure involves no training. The unsupervised learning
network presented in this discussion does not involve training
and is capable of learning immediately. Also, the network is
highly noise tolerant. The discussers have already applied this
kind of training for counterpropagation networks to various
problems (Rajasekaran 1997; Rajasekaran and Pai 1996, 1997;
Rajasekaran and Prameela, in press, 1998) and found it to be
suitable for complicated problems.
APPENDIX. REFERENCES
Rajasekaran, S. (1997). "Training free counterpropagation neural network
to pattern recognition in fabric defects." Textile Res. J., 67(6).
Rajasekaran, S., and Pai, G. A. V. (1996). "Training free counterpropa-
gation networks as fuzzy rule processors." Proc. Int. Con! on Neural
Information Processing, Hong Kong, China.
Rajasekaran, S., and Pai, G. A. V. (1997). "Training free counterpropa-
gation networks as static heteroassociative memories." Indian J.
Engrg. and Mat. Sci., New Delhi, India, 4,245-253.
Rajasekaran, S., and Prameela, K. S. (1998). "Computing with words
with fuzzy backpropagation and training free counterpropagation net-
works." J. Struct. Engrg., in press.
Closure by Abhijit Mukherjee4
The writer thanks the discussers for their interest in the pa-
per. The discussers argue that the network training can be
eliminated when the input and the synaptic weights are nor-
malized. The writer does not agree with this view. Through
training, the synaptic weights are updated continuously to pre-
serve in them a statistical mean of the input patterns. There-
fore, the synaptic weights continuously adapt to the input. This
is not possible when the training process is eliminated. The
illustrations presented by the discussers simply demonstrate
that the network has already trained (i.e., the synaptic weights
represent the classes properly). The training of the illustrations
presented is easy to the extent that the synaptic weights can
be guessed manually. This is due to a good separation between
the classes present. However, when the input needs to be clas-
sified with a finer resolution, training is mandatory. The writer
has demonstrated this in later research works (Mukherjee and
Kamat, in press, 1997).
The writer would like to indicate a few points in the dis-
cussers' note that may lead to confusion of the readers. In
illustration 1 the output vectors are presented as Y 10 Y2, and
Y 3' It is also indicated that the network yields those output
vectors. The output vectors are not necessary for the self-or-
ganizing network presented here. Neither does the network
·Assoc. Prof., Dept. of Civ. Engrg., Indian Inst. of Technol., Bombay
400 076, India.
184/ JOURNAL OF COMPUTING IN CIVIL ENGINEERING / JULY 1998
J. Comput. Civ. Eng., 1998, 12(3): 164-166
the input data into a number of classes. Therefore, the output
vectors presented as Y 10 Y2, and Y 3 are not required by the
(22) network. Moreover, in illustration 2, in the sentence "It may
be calculated that the two patterns that are distorted versions
of input vectors Xl and X 3 have their Hamming distances
'closer' to that of Xl and X3," Xl and X 3 should be replaced
(23) by X2 and X 3 •
APPENDIX. REFERENCE
(24) Mukherjee, A., and Kamat, R. (1997). "A self-organising neural network
in damage assessment of multi-storeyed building frames." J. Sound
and Vibration, in press.
(25)
CONSTRUCTABILITY ANALYSIS:
MACHINE LEARNING ApPROACIf'
Discussion by Yoram Reich4
The first 1997 issue of the Journal of Computing in Civil
Engineering includes two papers on the use of machine learn-
ing (ML) for two different purposes: improving understanding
(the paper under discussion) and making predictions (Wang
and Gero 1997). When creating knowledge by ML or in any
other way, it is critical to assess its quality (Reich 1995). This
assessment depends on the way one defines knowledge or on
the purpose of learning.
Artificial intelligence has developed two definitions of
knowledge: structural and functional (Reich 1995). According
to the structural definition, knowledge is whatever is repre-
sented in a system (e.g., facts, axioms, rules, causal relations,
etc.). Knowledge created by ML in comprehensible forms,
e.g., rules, may be assessed qualitatively, for example, by ex-
perts' interpretation of learned knowledge or by comparison
with expert-generated rules. This assessment can serve as the
evaluation of the ML program that created this knowledge.
This is the evaluation approach described by the authors.
According to the functional definition, knowledge is what a
system has that allows it to perform. Thus, knowledge cannot
be assessed directly, but rather, indirectly through observing
the performance of the system manipulating it. Knowledge
created by ML could be evaluated by assessing how it is used
to perform tasks. For example, learned knowledge could be
evaluated by comparing its predictive accuracy with some
baseline performance. This baseline may be another ML tech-
nique, other computational tools such as expert systems, de-
fault rules, or expert judgment.
The best assessment of a ML program is through assessing
the practical benefit of the deployed system. However, this
state has not yet been reached in the application of ML in civil
engineering (Reich 1997). Obviously a combination of all
these assessments is recommended. Thus, at the level of eval-
uation, the two aforementioned papers complement each other;
together, they give an opportunity to elaborate on the evalu-
ation of ML programs and correct several details found in
these papers. (A procedural reason requires that the discussion
'January 1997, Vol. 11, No. I, by Miroslaw Skibniewski, Tomasz
Arciszewski, and Kamolwan Lueprasert (Paper 11420).
·Sr. Lect., Dept. of Solid Mech., Mat., and Struct., Facu. of Engrg.,
Tel Aviv Univ., Ramat Aviv 69978, Israel. E-mail: yoram@eng.tau.ac.il.
 of these papers be split into two separate discussions; never-
theless, contentwise they should have been discussed together.)
The authors describe the use of ML to extract knowledge
about constructability. They demonstrate this by applying the
learning program INLEN to a database of 31 examples. They
create two rule sets using two modes, generalization and spe-
cialization, of the program. They report that the "rules were
found by practicing engineers as acceptable in the context of
examples used to produce them." This knowledge assessment
can be improved by following a careful procedure including
careful evaluation by experts augmented by quantitative anal-
ysis of the rules, as discussed later.
Qualitative expert evaluation is clearly subjective and has
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 10/23/18. Copyright ASCE. For personal use only; all rights reserved.
to be performed with care (Reich 1997). It is critical to per-
form it with several experts to get an intersubjective opinion.
If an evaluation involves comparing learned rules with other
knowledge, a particular setup should be followed in which one
group of experts evaluates the learned rules with respect to a
baseline and another group-serving as the control group-
evaluates another arbitrary set of rules compared to the same
baseline. The experts must not know to which of the groups
they belong. It is outside the scope of this discussion to explain
statistical experimental design in depth; nevertheless, without
careful attention to such design, results would be unreliable.
The discusser ran the learning program CN2 (Clark and Ni-
blett 1989) on the 31 examples to compare its performance
with INLEN. (I thank Peter Clark and Robin Boswell for the
access to CN2 and NewID.) CN2 employs the covering al-
gorithm used by INLEN (or AQI5). In addition, it uses en-
tropy measurement to assess the quality of rules, thus can han-
dle noise well. This ability could lead CN2 to produce rules
that are more general than those of INLEN. The rules extracted
by CN2 are given in Table 5. The presentation follows the
self-explanatory format of Table 3. In addition, the last column
states how many examples of the same decision a rule covers.
No rule covered examples of other decisions. In this sense,
the rules are an accurate reflection of the data.
Rules 1 and 4 are more general than their corresponding
rules in Table 3, whereas rule 5 is the same for both programs.
From the table, and as predicted, CN2 rules tend to be more
general than INLEN rules and still correct. A rule that covers
more examples is more "reliable" and interesting from an
understanding viewpoint. Given the small data set, the table
shows that there are two or at most three interesting rules: 1,
5, and 10. The rest cover too few examples to warrant assess-
ment.
When using ML for creating models (e.g., rules) that im-
prove understanding, it is fruitful to use several tools because
each provides a different perspective of the data (Reich et al.
1996). The utility of each model depends on its evaluation by
experts and by other assessments. One such assessment deals
with how well the rules reflect the data. This can be answered
quantitatively by executing a performance test. In order to ex-
plain it, the discusser executed these tests with CN2. The basic
test involves checking how the rules reconstruct the decisions
of the examples. This is a coverage or a resubstitution test that
produces an optimistic upper-bound estimation of accuracy.
The rules created by CN2 achieve 100% accuracy in this test.
Another common test used on ML programs, the hold-out,
uses about 70-80% of the examples for training and the rest
for testing. It can be pessimistic and is inaccurate for small
data sets (i.e., those having less than about 1,000 examples in
the testing set). The better tests are the k-fold cross-validation
(CV) or leave-one-out. These tests can be improved by using
the bootstrap method. See Reich (1997) for an elaboration on
this subject.
The rule set in Table 5 was created using parameters whose
combination, lest the type of rule error estimation, led to the
JOURNAL OF COMPUTING IN CIVIL ENGINEERING / JULY 1998/165
J. Comput. Civ. Eng., 1998, 12(3): 164-166
best performance in a leave-one-out experiment. The optimal
accuracy was 74.2% compared to the default rule, whose ac-
curacy is 45.2%, and compared to the pessimistic 64.0% with
0.15% standard deviation in a hold-out test with 100 iterations,
each using 22 training examples and nine for testing. Note that
when ML is used for the purpose of understanding, one can
use the optimal set of parameters or even use inferior para-
meters. [One cannot simply use the optimal parameters for the
performance purpose as done by Wang and Gero (1997).] The
guiding principle is improved understanding and not blind
model quality in a performance test. To illustrate, another rule
set created by using a different type of rule error estimation
achieved an accuracy of 87.1 % in the leave-one-out experi-
ment. Nevertheless, the rules created were very specific and
thus are not too interesting from the understanding purpose
viewpoint.
To further illustrate the usefulness of using multiple models
for better understanding, the discusser ran the decision tree
induction program NewID (Boswell 1990) on the same data
The performance of NewlD in a leave-one-out test was 67.7%
accuracy, suggesting that its models are "less accurate" than
CN2 on this data. The decision tree created (after simple
"postprocessing") was
NoSla
One:BeCha1
SlichgeReinf:ReRa
Lov:CoBeRa1
Lov: EXCELLENT 1
High: Good 1
Average: GOOD 1
AIIChgeReinf: GOOD 2
None: EXCELLENT 2
SameTvo:BeCha1
<> SlichgeReinf: POOR 3
SlichgeReinf: GOOD 4
DiffTvo:BeCha1
<> None: POOR 8
None: GOOD 2
None:ReRa
Lov: EXCELLENT 3
Average: GOOD 4
The number after the decision is the number of examples this
leaf covers. Interesting rules can be created by traversing the
tree from its root to some leaves. For example, the rule
IF (NoSla = DiffTwo) & (BeChal ( ) None)
THEN ConEva = POOR
covers eight examples, thus is fairly general. This rule does
not appear in Table 5 or Table 3. It might be interesting to ask
experts whether it is interesting. In summary, although we
would avoid using models that are poor in performance tests,
a "slightly worse" model from a performance viewpoint can
still give a different and complementary perspective, thus im-
proving understanding.
Compared to CN2 or NewID, it might be that INLEN per-
formance on a leave-one-out test is better, but this figure is
not provided. Therefore, it is hard to assess the suitability of
using INLEN to extract knowledge from the database. It is
also hard to assess the quality of the data Arciszewski et al.
(1995) discussed the complete process of using ML programs
for knowledge extraction including the process of collecting
 TABLE 5 . Rul•• Created by CN2
Decisions Conditions
CoBeRa1 CoBeRa2 BeCha1 BeCha2
(column-to- (column-to- NoSla NoWall (changes in (changes In Example
ConEva ReRa beam beam (number of (number of reinforcement reinforcement covered
Rule (construction (reinforcement reinforcement reinforcement slabs attached walls attached and size of and size of of same
number evaluation) ratio) ratio 1) ratio 2) to beam) to beam) left beam) right beam) decision
(1 ) (2) (3) (4) (5) (6) (7) (8) (9) (10)
I Poor High DiffI'wo S
2 Poor High One 3
3 Poor DiffI'wo 3
4 Poor High 3
S Good Average Low 6
6 Good High None SlichgeReinf 3
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 10/23/18. Copyright ASCE. For personal use only; all rights reserved.

7 Good Average One 2

8 Good Low SameTwo 3
9 Good Low High 2
10 Excellent Low One 4
11 Excellent Low None 2
Note: DiffI'wo =different slabs or walls on each Side; SameTwo = same slabs or walls on both sides; SlichgeReinf = slight changes in reinforcement.

examples, the evaluation of results, and the assessment of the
"quality" of the data. It is recommended that when using
ML programs, a similar process to the one discussed by
Arciszewski et al. (1995) or by Reich (1997) should be used.
One last minor comment: The authors confused COBWEB
with CLUSTER, which is also an unsupervised learning pro-
gram, in their discussion of BRIDGER (page 10).
APPENDIX. REFERENCES
Arciszewski, T., Michalski, R. S., and Dybala, T. (1995). "STAR meth-
odology-based learning about construction accidents and their preven-
tion." Automation in Constr., 4(1), 75-85.
Boswell, R. (1990). "Manual for NewID version 4.1." Tech. Rep. TIl
P21541RAB1412.3, Turing Institute, Glasgow, U.K.
Reich, Y. (1995). "Measuring the value of knowledge." Int. J. Human-
Compo Studies, 42(1), 3-30.
Reich, Y. (1997). "Machine learning techniques for civil engineering
problems." Microcomputers in Civ. Engrg., 12(4),295-310.
Reich, Y., Medina, M., Shieh, T.-Y., and Jacobs, T. (1996). "Modeling
and debugging engineering decision procedures with machine learn-
ing." J. Computing in Civ. Engrg., ASCE, 10(2), 157-166.
Wang, W., and Gero, J. S. (1997). "Sequence-based prediction in con-
ceptual design of bridges." J. Computing in Civ. Engrg., ASCB, 11(1),
37-43.
Closure by Tomasz Arciszewski5
The writers appreciate the comments of the discusser. He
correctly pointed out that it is critical to assess the quality of
the automatically acquired knowledge. However, the objective
of the paper was to "describe a novel approach to construct-
ability knowledge acquisition based on the use of machine
learning and to demonstrate its feasibility," not to produce any
specific domain-related knowledge and to verify its quality.
For all these reasons, the writers used only a very small num-
ber of examples. Thus, the use of statistical performance eval-
uation methods for such a small set of examples would simply
be questionable, at best. Therefore, the issue of knowledge
quality was only marginally addressed in the paper. The writ-
ers are aware, however, of the importance of this issue and it
is their pleasure to provide a response to the discussion. This
response is difficult, however, because the discusser mostly
refers his comments to a paper that at the time of writing is
still in print.
The writers hope that the discusser's and their own com-
'Assoc. Prof. of Urban Sys. Engrg., George Mason Univ., Fairfax, VA
22030-4444.
166/ JOURNAL OF COMPUTING IN CIVIL ENGINEERING / JULY 1998
ments about the evaluation of quality of engineering decision
rules will initiate a discussion about this critical subject for
the future of machine learning in engineering. The second
writer became involved in work on automated knowledge ac-
quisition in the mid-1980s. He soon realized that a formal and
objective evaluation of quality of acquired decision rules is the
key to the acceptance of learning systems by the engineering
community. For this reason, he developed in 1989 an initial
outline of a method for evaluation of a collection of examples!
decision rules in the context of case-based optimization (Ar-
ciszewski and Ziarko 1991). In 1992, he conducted studies of
performance-based evaluation of learning systems and of col-
lections of decision rules. This work resulted in a formal eval-
uation method presented in Arciszewski et al. (1992). The
method has been used extensively in various machine learning
experiments conducted at George Mason University dealing
with symbolic selective and constructive induction-based
learning systems during the period 1993-1996. The results of
this work, which clearly demonstrated advantages and limita-
tions of performance-based evaluation of quality of decision
rules, are presented in Szczepanik et al. (1996). Therefore, he
recently proposed a semantic evaluation of decision rules in
the context of the background knowledge understood as a hi-
erarchical system of concepts and their relationships in a given
domain. This method is presented in Arciszewski (1998).
In the context of the writers' experience in the area of qual-
ity evaluation of decision rules, they entirely agree with the
discusser's comments, which complement the paper and the
writers' own discussion of the acquired decision rules. How-
ever, one should be very careful in reaching definite conclu-
sions while working with only 31 examples. In this case, any
formal performance-based analysis of decision rules can ob-
viously be conducted only in the context of those examples.
Moreover, in the case of such a small number of examples the
use of statistical methods becomes quite controversial. There-
fore, if the objective is to learn about a given domain, one
should use a much larger number of examples of a balanced
nature within this domain. That would give one a chance to
acquire a larger, or better, collection of decision rules that
could be correctly evaluated using statistical methods.
The comment about using several tools to learn about a
given domain addresses a very complex issue of knowledge
acquisition. The writers agree with the discusser that using
various tools may produce complementary results in some
cases, particularly when significantly different tools are used,
for example tools producing decision rules and decision trees,
utilizing selective and constructed attributes, etc. However, the
recent experiments of Arciszewski et al. (1997) clearly dem-
onstrated that this may not be always the case. When two

J. Comput. Civ. Eng., 1998, 12(3): 164-166

 advanced symbolic selective learning systems are used, and
both are comparable in tenns of their sophistication and per-
formance measured by various empirical error rates, decision
rules produced can be surprisingly similar, if not identical,
even if the learning systems are based on entirely different
learning algorithms. In the case of the experiments referred to,
Arciszewski et al. (1997) used a system based on the theory
of rough sets (Pawlak 1982, 1991) and a system based on
STAR methodology (Michalski 1983; Michalski et al. 1986).
The results of learning conducted on various collections of
examples were nearly identical. It is admitted, however, that
in this particular case both systems were used with their de-
fault settings. As is known, more advanced learning systems
Downloaded from ascelibrary.org by Universidad Nacional De Ingenieria on 10/23/18. Copyright ASCE. For personal use only; all rights reserved.
allow their tuning to a given collection of examples in the
context of learning objectives. Therefore, both systems used
in the experiments could produce significantly different results,
if desired. In any case, the "discovery" of the similarity of
results when using different learning systems may be signifi-
cant in engineering for several reasons. First, it may indicate
that learning systems have already reached such a level of
maturity that their use in engineering is justified. Second, it
may suggest that the selection of a learning system for a given
task may not be as significant as previously thought. As is
known, many developers of learning systems claim unique fea-
tures and superior performance of their systems with respect
to the other systems available. These claims confuse potential
users of learning systems who are simply unable to conduct
extensive comparison tests before performing knowledge ac-
quisition. Consequently, engineers are reluctant to use learning
systems, and the progress in this area is unnecessarily delayed.
In any case, much more research has to be conducted on com-
parison of various learning systems before a full understanding
JOURNAL OF COMPUTING IN CIVIL ENGINEERING / JULY 1998/187
J. Comput. Civ. Eng., 1998, 12(3): 164-166
is reached of the effect of the underlying learning algorithms
and of tuning on the performance of a learning system.
Again, the writers express their sincere appreciation for the
discusser's stimulating discussion and hope that it will bring
about an increase in the attention of the research community
as well as of practicing engineers to the potential of machine
learning in solving civil engineering problems.
The writers appreciate the correction made by the discusser
regarding his research on machine learning in design knowl-
edge acquisition, conducted at Carnegie-Mellon University.
BRIDGER is based on the learning program COBWEB, and
not on CLUSTER as stated in the paper.
APPENDIX. REFERENCES
Arciszewski, T. (1998). "Engineering semantic evaluation of decision
rules." J. Intelligent and Fuzzy Sys., 5(3).
Arciszewski, T.• Ardayfio, D., and Doulamis, J. (1997). "Automated
knowledge acquisition in proactive design." Proc. ASME Engrg. Des.
Con/., American Society of Mechanical Engineers, New York, N.Y.,
1-11.
Arciszewski, T., Dybala, T., and Wnek, T. (1992). "A method for eval·
uation of learning systems." J. Knowledge Engrg. "Heuristics." 2(4),
22-31.
Arciszewski, T., and Ziarko, W. (1991). "Structural optimization: A case·
based approach." J. Computing in Civ. Engrg., ASeE, 5(2), 159-174.
Michalski, R. S., Hong, J., and Mozetic, I. (1986). "AQI5: Incremental
learning of attribute-based descriptions from examples, the method and
user's guide." Dept. of Camp. Sci. Rep., University of nlinois, Urbana,
Ill.
Pawlak, Z. (1991). Rough sets: Theoretical aspects of reasoning about
data. Kluwer Academic Publishers Group, Boston, Mass.
Szczepanik, w., Arciszewski, T., and Wnek, J. (1996). "Empirical com-
parison of selective and constructive induction." Engrg. Applications
of Artificial Intelligence. 9(6), 627 -639.