PDF Artificial Intelligence 5Th Raai Summer School Dolgoprudny Russia July 4 7 2019 Tutorial Lectures Gennady S Osipov Ebook Full Chapter

Artificial Intelligence 5th RAAI Summer
School Dolgoprudny Russia July 4 7

2019 Tutorial Lectures Gennady S.
Osipov
Visit to download the full and correct content document:
https://textbookfull.com/product/artificial-intelligence-5th-raai-summer-school-dolgopru
dny-russia-july-4-7-2019-tutorial-lectures-gennady-s-osipov/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Business Intelligence 5th European Summer School eBISS

2015 Barcelona Spain July 5 10 2015 Tutorial Lectures
1st Edition Esteban Zimányi
https://textbookfull.com/product/business-intelligence-5th-
european-summer-school-ebiss-2015-barcelona-spain-
july-5-10-2015-tutorial-lectures-1st-edition-esteban-zimanyi/
Business Intelligence Third European Summer School

eBISS 2013 Dagstuhl Castle Germany July 7 12 2013
Tutorial Lectures 1st Edition Esteban Zimányi (Eds.)
https://textbookfull.com/product/business-intelligence-third-
european-summer-school-ebiss-2013-dagstuhl-castle-germany-
july-7-12-2013-tutorial-lectures-1st-edition-esteban-zimanyi-eds/
Reasoning Web Declarative Artificial Intelligence 16th

International Summer School 2020 Oslo Norway June 24 26
2020 Tutorial Lectures Marco Manna
https://textbookfull.com/product/reasoning-web-declarative-
artificial-intelligence-16th-international-summer-
school-2020-oslo-norway-june-24-26-2020-tutorial-lectures-marco-
manna/
Business Intelligence 4th European Summer School eBISS

2014 Berlin Germany July 6 11 2014 Tutorial Lectures
1st Edition Esteban Zimányi
https://textbookfull.com/product/business-intelligence-4th-
european-summer-school-ebiss-2014-berlin-germany-
july-6-11-2014-tutorial-lectures-1st-edition-esteban-zimanyi/
Reasoning Web Semantic Interoperability on the Web 13th
International Summer School 2017 London UK July 7 11
2017 Tutorial Lectures 1st Edition Giovambattista Ianni
https://textbookfull.com/product/reasoning-web-semantic-
interoperability-on-the-web-13th-international-summer-
school-2017-london-uk-july-7-11-2017-tutorial-lectures-1st-
edition-giovambattista-ianni/
Engineering Trustworthy Software Systems: 5th

International School, SETSS 2019, Chongqing, China,
April 21–27, 2019, Tutorial Lectures Jonathan P. Bowen
https://textbookfull.com/product/engineering-trustworthy-
software-systems-5th-international-school-setss-2019-chongqing-
china-april-21-27-2019-tutorial-lectures-jonathan-p-bowen/
Artificial Intelligence XXXVI 39th SGAI International

Conference on Artificial Intelligence AI 2019 Cambridge
UK December 17 19 2019 Proceedings Max Bramer
https://textbookfull.com/product/artificial-intelligence-
xxxvi-39th-sgai-international-conference-on-artificial-
intelligence-ai-2019-cambridge-uk-
december-17-19-2019-proceedings-max-bramer/
Advances in Neural Computation Machine Learning and

Cognitive Research III Selected Papers from the XXI
International Conference on Neuroinformatics October 7
11 2019 Dolgoprudny Moscow Region Russia Boris
Kryzhanovsky
https://textbookfull.com/product/advances-in-neural-computation-
machine-learning-and-cognitive-research-iii-selected-papers-from-
the-xxi-international-conference-on-neuroinformatics-
october-7-11-2019-dolgoprudny-moscow-region-russia/
Lectures on the Geometry of Tensors Informal Notes for

the Nordfjordeid Summer School 2010 Giorgio Ottaviani
https://textbookfull.com/product/lectures-on-the-geometry-of-
tensors-informal-notes-for-the-nordfjordeid-summer-
school-2010-giorgio-ottaviani/
Gennady S. Osipov
Aleksandr I. Panov
Konstantin S. Yakovlev (Eds.)
Tutorial
LNAI 11866
Artificial Intelligence
5th RAAI Summer School
Dolgoprudny, Russia, July 4–7, 2019
Tutorial Lectures
Lecture Notes in Artificial Intelligence 11866
Subseries of Lecture Notes in Computer Science
Series Editors
Randy Goebel
University of Alberta, Edmonton, Canada
Yuzuru Tanaka
Hokkaido University, Sapporo, Japan
Wolfgang Wahlster
DFKI and Saarland University, Saarbrücken, Germany
Founding Editor
Jörg Siekmann
DFKI and Saarland University, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/1244
Gennady S. Osipov Aleksandr I. Panov
• •
Konstantin S. Yakovlev (Eds.)
Artificial Intelligence
5th RAAI Summer School
Dolgoprudny, Russia, July 4–7, 2019
Tutorial Lectures
123
Editors
Gennady S. Osipov Aleksandr I. Panov
Federal Research Center Federal Research Center
“Computer Science and Control” “Computer Science and Control”
Moscow, Russia Moscow, Russia
Konstantin S. Yakovlev
Federal Research Center
“Computer Science and Control”
Moscow, Russia
ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Artificial Intelligence
ISBN 978-3-030-33273-0 ISBN 978-3-030-33274-7 (eBook)
https://doi.org/10.1007/978-3-030-33274-7
LNCS Sublibrary: SL7 – Artificial Intelligence
© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, expressed or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
Cover illustration: Illustration of different types of conflicts, taken from Stern et al. [37]. LNAI 11866,
p. 97. Used with permission.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
The 5th RAAI Summer School on Artificial Intelligence was held in Dolgoprudy,
Russia at the Moscow Institute of Physics and Technology (MIPT) during July 4–7,
2019. MIPT is one of the leading universities in Russia, especially renowned for its
achievements in the fields of physics, mathematics, and computer sciences. The school
was organized by the Russian Association for Artificial Intelligence which is a major
academic non-profit organization in the field of AI in Russia.
More than 100 participants from all over the world (mostly from Russia, but also
from Germany, Sweden, China, Turkey, Armenia, Syria, and Iran) took part in a
four-day marathon comprised of lectures, workshops, hackathons, industry sessions,
etc.
This tutorial book is composed of the selected tutorials by the invited speakers of
RAAI Summer School 2019 and of the best students’ papers. In total 20 student
submissions were received and only 5 of them were chosen by the international
Program Committee to be included in the book.
We appreciate the financial support of the school’s sponsors (i.e.: MIPT, Yandex,
Huawei, AimTech, NLMK, and Tinkoff) without which it would not have been
possible to invite top AI experts to deliver the talks and make the participation free for
all students.
July 2019 Gennady S. Osipov

Aleksandr I. Panov
Konstantin S. Yakovlev
Organization
Program Committee
Gennady S. Osipov President of RAAI, head of Artificial Intelligence
(Co-chair) Research Institute of Federal Research Center
“Computer Science and Control” of the Russian
Academy of Sciences, Russia
Ricardo Ribeiro Gudwin University of Campinas, Brazil
(Co-chair)
Alexey Averkin Federal Research Center “Computer Science and
Control” of the Russian Academy of Sciences,
Russia
Ildar Batyrshin Instituto Politecnico Nacional, Mexico
Mikhail Burtsev Moscow Institute of Physics and Technology, Russia
Vadim Vagin National Research University MPEI, Russia
Michal Valko Inria Lille, France
Tamás Gergely Applied Logic Laboratory, Hungary
Vladimir Golenkov Belarusian State University of Informatics and
Radioelectronics, Belarus
Valeria Gribova Institute of Automation and Control Processes of the
Far Eastern Branch of RAS, Russia
Alexandr Eremeev National Research Nuclear University MPEI, Russia
Valery Karpov NRC “Kurchatov Institute”, Russia
Namkug Kim University of Ulsan, South Korea
Sergey Kovalev Rostov State Transport University, Russia
Vladik Kreinovich University of Texas at El Paso, USA
Sergey O. Kuznetsov Higher School of Economics in Moscow, Russia
Oleg Kuznetsov Trapeznikov Institute of Control Sciences, Russia
Hermann Ney RWTH Aachen University, Germany
Evgeny Osipov Luleå University of Technology, Sweden
Vladimir Pavlovsky Keldysh Institute of Applied Mathematics, Russia
Boris Palyukh Tver State Technical University, Russia
Witold Pedrycz University of Alberta, Canada
Andrei Raigorodskii Moscow Institute of Physics and Technology, Russia
Galina Rybina National Research Nuclear University MEPhI, Russia
Ruslan Salakhutdinov Carnegie Mellon University, USA
Vadim Stefanuk Institute for Information Transmission Problems
of RAS, Russia
Valery Tarasov Bauman University, Russia
Alexander Tulupyev St. Petersburg Institute for Informatics and Automation
of RAS, Russia
viii Organization
Andrey Filchenkov ITMO University, Russia

Igor Fominykh National Research University MPEI, Russia
Vladimir Khoroshevsky Federal Research Center “Computer Science and
Control” of the Russian Academy of Sciences,
Russia
Roni Stern Ben Gurion University of the Negev, Israel
Organizing Committee
Aleksandr I. Panov Artificial Intelligence Research Institute of Federal
(Co-chair) Research Center “Computer Science and Control”
of the Russian Academy of Sciences, Russia
Andrei Raigorodskii Moscow Institute of Physics and Technology, Russia
Konstantin Yakovlev Artificial Intelligence Research Institute of Federal
Research Center “Computer Science and Control”
Alena Suvorova Higher School of Economics in Saint-Petersburg,
Russia
Nikolay Bazenkov Trapeznikov Institute of Control Sciences, Russia
Elena Fontalina National Research Nuclear University MEPhI, Russia
Maria Koroleva Bauman University, Russia
Margarita Suvorova Artificial Intelligence Research Institute of Federal
Research Center “Computer Science and Control”
Contents
Tutorial Papers
Hybrid Intelligent Systems Based on Fuzzy Logic and Deep Learning . . . . . . 3

Alexey Averkin
Data Science: Similarity, Dissimilarity and Correlation Functions . . . . . . . . . 13

Ildar Z. Batyrshin
Mathematical Foundation of Cognitive Computing Based

Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Tamás Gergely and László Ury
A Review of Motivational Systems and Emotions in Cognitive

Architectures and Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Ricardo R. Gudwin
Selected Challenges in Grammar-Based Text Generation

from the Semantic Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Simon Mille
Multi-Agent Path Finding – An Overview . . . . . . . . . . . . . . . . . . . . . . . . . 96

Roni Stern
Young Scientist School Papers
The Use of Reinforcement Learning in the Task of Moving Objects

with the Robotic Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Ermek E. Aitygulov
Ontology Models in Intelligent System Engineering: A Case

of the Knowledge-Intensive Application Domain. . . . . . . . . . . . . . . . . . . . . 127
Karina A. Gulyaeva and Irina L. Artemieva
Automated Acquisition, Representation and Processing of Temporal

Knowledge in Dynamic Intelligent Systems . . . . . . . . . . . . . . . . . . . . . . . . 140
Galina V. Rybina and Elena S. Fontalina
Natural Language Processing with DeepPavlov Library and Additional

Semantic Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Oleg Sattarov
x Contents
Toward Faster Reinforcement Learning for Robotics:

Using Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Ali Younes and Aleksandr I. Panov
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Tutorial Papers
Hybrid Intelligent Systems Based
on Fuzzy Logic and Deep Learning
Alexey Averkin(&)
Federal Research Centre of Informatics and Computer Science of RAS, Moscow,

Vavilova, 42, Moscow, Russia
averkin2003@inbox.ru
Abstract. The purpose of this lecture is to establish the fundamental links

between two important areas of artificial intelligence - fuzzy logic and deep
learning. This approach will allow researchers in the field of fuzzy logic to
develop application systems in the field of strong artificial intelligence, which
are also of interest to specialists in the field of machine learning. The lecture also
examines how neuro-fuzzy networks make it possible to establish a link between
symbolic and connectionist schools of artificial intelligence. A lot of methods of
rule extraction from neural networks are also investigated.
Keywords: Deep learning Neural networks Rule extraction Convolutional

neural network Machine learning Artificial intelligence
1 Introduction
This lection introduces the terms and definitions of machine learning that are relevant
to the context of extracting rules from classical and deep neural networks. It includes
the problem of classification as a whole, as well as rule-based teaching methods and
neural networks. Then, we will look at the current state of rule extraction from neural
networks. Here we define the problem as well as the main approaches to its solution
and present some of the existing rules extraction algorithms. The last part discusses
specific problems when working with deep neural networks. At this stage, we also
propose some algorithms that can successfully extract rules from these more complex
neural networks.
Artificial Neural Networks (ANN) are widely known parallel computing models
that exhibit excellent behavior in solving complex problems of artificial intelligence.
However, many researchers refuse to use them due to their being a “black box”. This
means that determining why a neural network makes a specific decision is a difficult
task.
This is a significant drawback, since it is difficult to trust the reliability of the
network that solves real problems without the ability to make acceptable decisions. For
example, this is the case in critical, in terms of safety, applications where hidden failure
can lead to life-threatening actions or huge economic losses.
In addition, studying how neural networks extract, store and transform knowledge
can be useful for future machine learning methods. For example, increasing the
transparency of neural networks can help detect the so-called hidden dependencies that
G. S. Osipov et al. (Eds.): Artificial Intelligence, LNAI 11866, pp. 3–12, 2019.
https://doi.org/10.1007/978-3-030-33274-7_1
4 A. Averkin
are not present in the input data, but appear as a result of their integration into the
neural network.
To overcome this lack of neural networks, researchers came up with the idea of
extracting rules from neural networks, which can became a bridge between symbolic
and connectionist models of knowledge representation in artificial intelligence.
Most authors focus on extracting the most understandable rules, and at the same
time they should mimic the behavior of the neural network as precisely as possible,
right up to an isomorphic representation of fuzzy rules in the form of a neuro-fuzzy
system. Since 1992, since Chang’s doctoral thesis on neuro-fuzzy networks, much
work has been done in this area, which ended with the creation of the direction of soft
computing. Since then, many methods for extracting rules from neural networks have
been developed and evaluated, and excellent results have been obtained for many
approaches.
However, despite the fact that there are quite a few available algorithms, none of
them has ever been explicitly tested in deep neural networks. In addition, most authors
focus on networks with only a small number of hidden layers.
Only in the last few years pioneering work has appeared on the analysis of specific
methods for extracting rules from deep-seated networks and new algorithms are pre-
sented that are capable of performing this task.
2 Methods for Extracting Rules from the Neural Network
In artificial intelligence, neural networks and rule-based learning methods are two
approaches to solving classification problems. Both methods are known variants of
studying models that predict classes for new data. For many tasks, NN rules-based
teaching methods excel in accuracy.
However, neural networks have one major drawback: the ability to understand what
a trained concept models, NN is not as strong as for rule based approaches. The
concepts learned by neural networks are difficult to understand because they are rep-
resented using a large set of parameters [1].
Increasing the transparency of neural networks by extracting rules has two main
advantages. First, it gives the user some insight into how the neural network uses input
variables to make a decision—and can even reveal hidden functions in NN when the
rules are used to explain individual neurons. Identification of particularly important
attributes or identification of the causes of neural network errors can be part of the
understanding. Trying to make opaque neural networks more understandable, methods
for extracting rules eliminate the gap between accuracy and clarity [2–4].
A more comprehensible form is required if, for example, a neural network is to be
used in safety-critical applications, such as aircraft and power plants. In these cases, it
is extremely important that the system user have the opportunity to check the output of
the artificial neural network under all possible input conditions [5].
To formalize the task of extracting rules from a neural network, we give Craven’s
definition: “Given the trained neural network and the data on which it was trained,
create a description of the network hypothesis that is understandable, but comes close
to the network prediction behavior” [6].
Hybrid Intelligent Systems Based on Fuzzy Logic and Deep Learning 5
To distinguish between different approaches for extracting rules from neural net-
works, Andrews introduced the widely used multidimensional taxonomy [5]. The first
dimension they describe is a form (for example, IF-THEN rules or a variant of fuzzy
production rules).
The third dimension is the quality of the rules extracted. Since quality is a broad
term, it is divided into several criteria, namely, accuracy, fidelity, consistency and
comprehensibility. While accuracy measures the ability to “correctly classify previ-
ously unseen examples”, validity measures the degree to which rules can “imitate” the
behavior of a neural network well [2].
The second dimension is called transparency and describes the strategy followed by
the algorithm for extracting rules. If the method uses a neural network only as a black
box, regardless of the architecture of NN, we call the pedagogical approach. If, instead,
the algorithm takes into account the internal structure of the neural network, we call this
approach decompositional. If the algorithm uses components of both pedagogical and
decomposition methods, then this approach is called eclectic.
The third dimension is the quality of the rules extracted. Since quality is a broad
term, it is divided into several criteria, namely, accuracy, fidelity, consistency and
comprehensibility. While accuracy measures the ability to correctly classify previously
unseen examples, validity measures the degree to which rules can imitate the behavior
of a neural network well [2].
Fidelity can be considered as accuracy relative to the output of NN. Consistency
can only be measured when the rule extraction algorithm involves learning the neural
network instead of processing the already trained NN: The extracted rule set is con-
sidered consistent when the neural network generates rule sets that correctly classify
test data for different training sessions. Comprehensibility is considered here as a
measure of the size of the rules, that is, short and few rules are considered more
understandable [5].
In this review we will focus only on the three criteria described. In accordance with
[7], we focus on methods that do not impose special requirements on how the neural
network was trained before the rules were extracted. In addition, only algorithms that
are capable of extracting rules from direct propagation neural networks, despite any
other characteristics of the architecture, are analyzed. According to [3] we want the
algorithm to offer a high level of generality.
Let us analyze some methods for extracting rules that meet the above character-
istics. We start with decomposition approaches. As mentioned earlier, decomposition
approaches for extracting rules from neural networks operate at the neuron level.
Usually, the decomposition method analyzes each neuron, and forms rules that imitate
the behavior of this neuron. For various reasons, we do not take into account all
available decomposition approaches in the subsequent review. We consider here the
KT algorithm, Tsukimoto’s approach and rule extraction through decision tree
induction (CRED algorithm).
The KT algorithm was one of the first decomposition approaches for extracting
rules from neural networks was presented in [29]. The KT algorithm describes each
neuron (layer by layer) with the IF-THEN rules by heuristically searching for com-
binations of input attributes that exceed the threshold of the neuron. The rewrite
module is used to obtain rules that refer to the original input attributes, and not to the
6 A. Averkin
outputs of the previous level. To find suitable combinations, the KT method applies a
search on the tree, that is, a rule (represented as a node in the tree) at this level generates
its child nodes by adding an additional available attribute [8]. In addition, the algorithm
uses a number of heuristics to stop the growth of a tree in situations where further
improvement is impossible. The algorithm searches for both confirming and non-
confirming rules. These are the rules that predict the triggering of a neuron if there is a
certain input configuration.
Polinomial Tsukimoto’s approach to extracting rules from a neural network is very
similar to the KT method. It also uses a layered decomposition algorithm to extract the
IF-THEN rules for each neuron, and also monitors the strategy for finding input
configurations that exceed the threshold of the neuron. The main advantage of the
Tsukimoto method is its computational complexity, which is polynomial, while the KT
method is exponential [9]. The algorithm achieves polynomial complexity by searching
for relevant terms using the space of multilinear functions. In the second stage, these
terms are used to create IF-THEN rules. Subsequently, if any, training data is used to
improve the accuracy of the rules. In the last step, the Tsukimoto algorithm attempts to
optimize clarity by removing non-essential attributes from the rules.
Another method for extracting rules through decision tree induction was introduced
in [10]. Their CRED algorithm converts each output unit of a neural network into a
solution, where tree nodes are tested using nodes of a hidden layer, and leaves represent
a class. After this, intermediate rules are extracted from this step. Then for each split
point used in these rules, another decision tree is created using split points on the input
layer of the neural network. In the new trees, the leaves do not directly choose the class.
Extracting the rules from the second decision tree leads us to the description of the state
of hidden neurons, consisting of input variables. As a final step, intermediate rules that
describe the output layer through the hidden layer and those that describe the hidden
layer based on the inputs of the neural network are replaced. Then they are combined
into construction rules that describe the output of the neural network based on its input
data.
The next group of pedagogical approaches of rule extraction based on validity
interval analysis, approaches for rule extraction using sampling and rule extraction by
reverse engineering the neural network.
Pedagogical approaches, as opposed to decomposition, do not take into account the
internal structure of the neural network. The motive in pedagogical approaches is to
treat trained NN at the lowest possible level of granularity, that is, as a whole or,
alternatively, as a black border [11]. Their idea is to extract the rules by directly
matching the input data with the output data [12]. More formally, we can assume that
pedagogical approaches have access only to the function of the neural network. This
function returns the output of the neural network for random input, but offers no
understanding of the internal structure of NN or any weights (except for the number of
inputs and outputs in NN). Having NN, this class of algorithms tries to find coherence
between possible input variations and outputs created by the neural network, while
some of them use specified training data, and some do not. As in the previous section,
we will not discuss all possible algorithms, but only briefly go over the main ones.
Rule extraction based on interval analysis approach uses the interval confidence
analysis (VIA), a kind of sensitivity analysis, to extract rules that mimic the behavior of
a neural network [13]. The main idea of this method is to find the input intervals in
which the output signal NN is stable, that is, the predicted class is the same for slightly
changing input configurations. As a result, VIA provides the basis for reliably correct
rules.
Retrieving rules using sampling represents several methods that follow more or less
the same strategy for extracting rules from a neural network using sampling, that is,
they create an extensive set of data as a basis for extracting rules. After that, the
selected data set is submitted to a standard learning algorithm for generating rules that
simulate the behavior of a neural network. In [2] it is proved that the use of sample data
exceeds the use of training data in the problems of extracting rules.
One of the first methods that followed this strategy was the Trepan algorithm [14].
Trephine works very much like C4.5 by searching for split points on training data for
individual instances of different classes. The main differences from C4.5 are the best
strategy for expanding the tree structure, additional split points in the M-of-N style and
the ability to choose additional learning examples at deeper points of the tree. As a
result, the algorithm also creates a decision tree, which, however, can be transformed
into a set of rules, if necessary.
Another of these very general pedagogical approaches that use sampling to extract
rules from the neural network is presented in [13]. The algorithm, called Binarized
Input-Output Rule Extraction (BIO-RE), is capable of processing only NN with binary
or binarized input attributes. BIO-RE creates all possible input combinations and
requests them from the neural network. Using the NN output, a truth table is created for
each example. From the truth table it is just as easy to go to the rules, if necessary.
ANN-DT is another decision-based sampling method for describing the behavior of
a neural network [14]. The overall algorithm is based on CART with some variations in
the initial implementation. ANN-DT uses the sampling method to expand the training
set so that most of the training sample is still representative. This is “achieved using the
nearest neighbor method, in which the distance from the sample point to the nearest
point in the training data set is calculated” [14] and compared with the reference value.
The idea of creating a large set of instances at the first stage is also implemented by
the STARE algorithm [15]. Like BIO-RE, STARE also forms extensive truth tables for
learning. The advantage of STARE is its ability not only to handle binary and discrete
attributes, but also to work with continuous input data. For the formation of truth tables,
the algorithm rearranges the input data, while for each continuous attribute “it is
necessary to sample it over the entire range of values with a high frequency”.
The last pedagogical approach using a sample of educational data that we want to
briefly present here is KDRuleEx [4]. Like Trepan, the Sethi algorithm also generates
additional learning cases where the basis for the following separation points is too
small. KDRuleEx uses a genetic algorithm to create new training examples. The
technique leads to a decision table that can be converted, for example, into IF-THEN
rules, if desired.
Eclectic approach are the methods for extracting rules include elements of both
pedagogical and decompositional, then such methods are known as eclectic [3]. In
particular, eclectic approaches use knowledge of the internal architecture and/or weight
vectors in the neural network to complement the symbolic learning algorithm [5].
8 A. Averkin
The fast retrieval of rules from a neural network approach includes the FERNN
approach, which first tries to identify the corresponding hidden neurons, as well as the
corresponding inputs to the network. For this step, a decision tree is constructed using
the well-known algorithm C4.5. The rule extraction process leads to the generation of
M-of-N and IF-THEN rules. Having a set of properly classified teaching examples,
FERNN analyzes the activation values of each hidden unit. For each hidden unit,
activation values are sorted in ascending order. Then use the C4.5 algorithm to find the
best split point to form the decision tree. The most interesting from the point of view of
this study is the extraction of rules using neuro-fuzzy models On the other hand,
systems based on fuzzy rules (FRBS), developed using fuzzy logic, have become a
field of active research over the past few years. These algorithms have proven their
strengths in tasks such as managing complex systems, creating fuzzy controls. The
relationship between both worlds (ANN and FRBS) has been carefully studied. Indeed,
this is a close relationship, since equivalence results were obtained [17].
This link gives two immediate and important conclusions. First, we can apply what
was discovered for one of the models to the other. Secondly, we can translate the
knowledge embedded in the neural network into a more cognitively acceptable lan-
guage - fuzzy rules. In other words, we get a clear interpretation of neural networks
[18–20].
3 Extracting Rules from Deep Neural Networks
Since 2012, the revolution began networks of deep learning. Consider one of the first
and probably the most cited works in convolutional NN- Alexnet - it has 7 hidden
layers, 650,000 neurons, 60,000,000 parameters. Studied at 2 GPU for 1 week (Pic-
ture 1). Where do we get enough images to train her?
Picture 1. Alexnet - first deep learning champion in image recognition
In 2010 dataset Imagenet has appeared. The emergence of Imagenet brought the
learning of neural networks to a whole new level. Parallel rapidly developed computing
power, which led computer vision to the kind that we know and love it now. Since
2010, the annual Imagenet competition has also been held, where for the first time in
2012, the Alexnet convolutional neural network won and, since then, the National
Assembly has not lost its positions. The last winner, the National Assembly presented
by scientists from China, contained 269 layers (Picture 2).
Picture 2. Dataset Imagenet.
To semantic intepretation of the deep learning blackbox neuro-fuzzy networks can

be use used instead of full connection layer (Fig. 3). For example, ANFIS (adaptive
neuro-fuzzy system) [25] is a multilayer feed forward network. This architecture has
five layers such as fuzzy layer, product layer, normalized layer, de - fuzzy layer and
total output. The fixed nodes are represented by circle and the node s represented by
square are the adapted nodes. ANFIS gives the advantages of the mixture of neural
network and fuzzy logic.
Fig. 3. Structure of ANFIS model for time series forecasting

10 A. Averkin
The aim of mixing fuzzy logic and neural networks is to de-sign an architecture,
which uses a fuzzy logic to show knowledge in fantastic way, while the learning nature
of neural network to maximize its parameters. ANFIS put forward by Jang in 1993
integrate the advantages of both neural network and fuzzy systems, which not only
have good learning capability, but can be interpreted easily also. ANFIS has been used
in many applications in many areas, such as function approximation, intelligent control
and time series prediction.
A hypothetical system can be created using two components [26]. The first is deep
learning feature generation which can be used to create representative features from
text directly. The deep learning system would initially be trained on unlabeled data.
Once these features are extracted from the deep learning system, they will be integrated
into fuzzy-inference systems. These systems can incorporate both the features detected
from the deep learning as well as subjective information from an analysts as a method
of biasing the system. These two pieces together can be used for classification pur-
poses. The final system would therefore be able to report both classification results and
the specific features and rules that were activated for the system to arrive at its con-
clusion. Additionally, the final system could be further biased by an analyst as a form
of feedback.
In previous part has shown that there is a wide variety of algorithms to describe a
one-hidden-layer neural network’s behavior by rules.. In [21] it was made a first step in
investigating the problem of extracting rules from deep neural networks. DNNs have a
few characteristics that, compared to one-hidden-layer neural networks, can complicate
the rule extraction process. For deep neural networks, the rule extraction problem is
very complex. We must not only to explain an output’s behaviour, but also to describe
the behaviour of a neuron in a hidden layer. In [21] authors give an overview of how
decompositional rule extraction algorithms like DeepRED or pedagogical approaches
can be used to analyse and explain the behaviour of a DNN.
Very interesting approach is suggested in [22, 23], where the author established a
fundamental connection between two important fields in artificial intelligence i.e. deep
learning and fuzzy logic. He shows, how deep learning could benefit from the com-
parative research by re-examining many trail-and-error heuristics in the lens of fuzzy
logic, and consequently, distilling the essential ingredients with rigorous foundations.
The author proposed deep generalized hamming network (GHN) as such not only lends
itself to rigorous analysis and interpretation within the fuzzy logic theory but also
demonstrates fast learning speed, well-controlled behaviour and state-of-the-art per-
formances on a variety of learning tasks. In [24] it is presented another approach for
incorporating such rule based methodology into neural networks by embedding fuzzy
inference systems into deep learning networks.
4 Conclusion
The study of fuzzy logic culminated in the end of the 20th century, and since then has
begun to decrease [27]. This decrease can be partly attributed to the lack of results in
machine learning. Extracting rules is one way to help understand neural networks.
These studies will pave the way for fuzzy logic researchers to develop applications in
artificial intelligence and solve complex problems that are also of interest to the
machine learning community. Experience and knowledge in the field of fuzzy logic are
well suited for modeling ambiguities in big data, modeling uncertainty in the repre-
sentation of knowledge and providing transmission training with non-inductive infer-
ence, etc.
References
1. Craven, M., Shavlik, J.W.: Using sampling and queries to extract rules from trained neural
networks. In: ICML, pp. 37–45 (1994)
2. Johansson, U., Lofstrom, T., Konig, R., Sonstrod, C., Niklasson, L.: Rule extraction from
opaque models–a slightly different perspective. In: 5th International Conference on Machine
Learning and Applications. ICMLA 2006, pp. 22–27 (2006)
3. Craven, M., Shavlik, J.: Rule extraction: where do we go from here. In: University of
Wisconsin Machine Learning Research Group Working Paper, pp. 99–108 (1999)
4. Sethi, K.K., Mishra, D.K., Mishra, B.: KDRuleEx: a novel approach for enhancing user
comprehensibility using rule extraction. In: 2012 Third International Conference Intelligent
Systems, Modelling and Simulation (ISMS), pp. 55–60 (2012)
5. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting
rules from trained artificial neural networks. Knowl.-Based Syst. 8(6), 373–389 (1995)
6. Craven, M.W.: Extracting comprehensible models from trained neural networks. Ph.D.
thesis, University of Wisconsin-Madison (1996)
7. Thrun, S.: Extracting provably correct rules from artificial neural networks. Technical report,
University of Bonn, Institut für Informatik III (1993)
8. Fu, L.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 24(8), 1114–
1124 (1994)
9. Tsukimoto, H.: Extracting rules from trained neural networks. IEEE Trans. Neural Networks
11(2), 377–389 (2000)
10. Sato, M., Tsukimoto, H.: Rule extraction from neural networks via decision tree induction.
In: International Joint Conference on Neural Networks. Proceedings. IJCNN 2001, vol. 3,
pp. 1870–1875 (2001)
11. Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The truth will come to light: directions
and challenges in extracting the knowledge embedded within trained artificial neural
networks. IEEE Trans. Neural Networks 9(6), 1057–1068 (1998)
12. .
13. Thrun, S.: Extracting rules from artificial neural networks with distributed representations.
In: Advances in Neural Information Processing Systems, pp. 505–512 (1995)
14. Craven, M.W., Shavlik, J.W.: Extracting tree-structured representations of trained networks.
In: Advances in Neural Information Processing Systems, pp. 24–30 (1996)
15. Taha, I.A., Ghosh, J.: Symbolic interpretation of artificial neural networks. IEEE Trans.
Knowl. Data Eng. 11(3), 448–463 (1999)
16. Towell, G.G., Shavlik, J.W.: Extracting refined rules from knowledge-based neural
networks. Mach. Learn. 13(1), 71–101 (1993)
17. Setiono, R., Leow, W.K.: FERNN: an algorithm for fast extraction of rules from neural
networks. Appl. Intell. 12(1–2), 15–25 (2000)
18. Averkin, A., Yarushev, S.: Hybrid neural networks for time series forecasting. In:
Kuznetsov, S.O., Osipov, G.S., Stefanuk, V.L. (eds.) RCAI 2018. CCIS, vol. 934, pp. 230–
239. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00617-4_21
12 A. Averkin
19. Pilato, G., Yarushev, S.A., Averkin, A.N.: Prediction and detection of user emotions based
on neuro-fuzzy neural networks in social networks. In: Abraham, A., Kovalev, S., Tarassov,
V., Snasel, V., Sukhanov, A. (eds.) IITI’18 2018. AISC, vol. 875, pp. 118–125. Springer,
Cham (2019). https://doi.org/10.1007/978-3-030-01821-4_13
20. Averkin, G.P., Yarushev, S.A.: An approach for prediction of user emotions based on
ANFIS in social networks. In: Second International Scientific and Practical Conference
Fuzzy Technologies in the Industry, FTI 2018–CEUR Workshop Proceedings, pp. 126–134
(2018)
21. Zilke, J.R., Loza Mencía, E., Janssen, F.: DeepRED – rule extraction from deep neural
networks. In: Calders, T., Ceci, M., Malerba, D. (eds.) DS 2016. LNCS (LNAI), vol. 9956,
pp. 457–473. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46307-0_29
22. Fan, L.: Revisit fuzzy neural network: demystifying batch normalization and ReLU with
generalized hamming network. In: NIPS 2017 (2017)
23. Fan, L.: Revisit Fuzzy Neural Network: Bridging the Gap between Fuzzy Ljgic and Deep
Learning. Technical Report (2017)
24. Bonanno, D., Nock, K., Smith, L., Elmore, P., Petry, F.: An approach to explainable deep
learning using fuzzy inference. In: Proceedings of the SPIE 10207, Next-Generation
Analyst V, 102070D (2017)
25. Jang, S.R.: ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man
Cybernet. 23, 665–685 (1992)
26. Bonanno, D., Nock, K., Smith, L., Elmore, P., Petry, F.: An approach to explainable deep
learning using fuzzy inference. In: Hanratty, T.P., Llinas, J. (eds.) Next-Generation
Analyst V. Proceedings of the SPIE, vol. 10207 (2017)
27. Goodfellow, I., et al.: Generative ad- versarial nets. In: Ghahramani, Z., Welling, M., Cortes,
C., Lawrence,N. D., Weinberger, K.Q., éditeurs: Advances in Neural Information Processing
Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014)
Data Science: Similarity, Dissimilarity
and Correlation Functions
Ildar Z. Batyrshin(&)
Instituto Politécnico Nacional, Centro de Investigación en Computación,

Av. Juan de Dios Bátiz S/N, Nueva Industrial Vallejo,
07738 Ciudad de México, CDMX, Mexico
batyr1@gmail.com
Abstract. The lecture presents a new, non-statistical approach to the analysis

and construction of similarity, dissimilarity and correlation measures. The
measures are considered as functions defined on an underlying set and satisfying
the given properties. Different functional structures, relationships between them
and methods of their construction are discussed. Particular attention is paid to
functions defined on sets with an involution operation, where the class of
(strong) correlation functions is introduced. The general methods constructing
new correlation functions from similarity and dissimilarity functions are con-
sidered. It is shown that the classical correlation and association coefficients
(Pearson’s, Spearman’s, Kendall’s, Yule’s Q, Hamann) can be obtained as
particular cases.
Keywords: Similarity measure Pearson’s product-moment correlation

Spearman’s rank correlation Kendall’s rank correlation Yule’s Q
1 Introduction
Data Science is the buzzword of the last decade. One of the goals of Data Science is to
extract knowledge, insights and usable information from various types of data. Data
Science courses typically include the topics on statistical data analysis, machine
learning methods and data mining. Correlation, association, similarity, relationship and
interestingness coefficients or measures play an important role in data analysis, clas-
sification tasks and data mining. Dozens of such measures have been created for
various types of data, and articles on these indicators are cited in hundreds and
thousands of papers [3, 13, 14, 20, 23–25, 27]. Such measures are defined for
dichotomous or real valued variables measured for sampling units, for rankings, binary
or real valued vectors of attributes, time series, fuzzy sets of different types, proba-
bilistic distributions, images, rating profiles etc. These measures used in information
retrieval, data classification, machine learning, analysis of relationships and decision
making in ecology, computational linguistics, image and signal processing, financial
data analysis, bioinformatics and social sciences. Often, measures introduced for one
type of data give misleading results when they are used for analysis of relationships
between data of another type [8]. There is a need in construction of a general theory of
similarity, correlation and association measures, which will be able to analyze general

G. S. Osipov et al. (Eds.): Artificial Intelligence, LNAI 11866, pp. 13–28, 2019.
https://doi.org/10.1007/978-3-030-33274-7_2
14 I. Z. Batyrshin
properties of these measures independent on domain or for large class of data types.
Such a theory will make it possible to transfer the methods of constructing association
measures from one domain to another and propose such measures for new data types.
The first steps in construction of the general theory of similarity, dissimilarity and
correlation measures have been done in [7, 9, 10] where the fundamental results on
definition and construction of association measures (correlation coefficients) on sets
with involution operations have been proposed. It was shown that many known cor-
relation coefficients can be considered as functions defined on an underlying set with
involution operation and satisfying a simple set of properties. The general methods of
construction of these functions from suitable similarity or dissimilarity functions have
been proposed. These results can be considered as an alternative solution of the
Kendall’s problem of construction of general correlation coefficient [22]. Kendall
proposed a formula that gives as particular cases Pearson’s product-moment correla-
tion, Spearman’s and Kendall’s rank correlation coefficients but it was not clear how to
obtain from this formula other known association and correlation coefficients and how
to build correlation coefficients for other domains. The approach proposed in [7, 9, 10]
gives possibility to construct the classical correlation coefficients that can be obtained
from Kendall’s formula, the Yule’s Q and Hamann coefficients together with some
other association coefficients for binary data and gives a tool for constructing new
correlation coefficients on new domains.
Similarity, dissimilarity and correlation measures or coefficients are considered in
this lecture as functions defined on an underlying set X and satisfying some reasonable
sets of properties. Generally, these functions can be considered as association func-
tions, measuring (may be non-statistical) associations or relationships between objects.
The methods of construction of such functions, the transformations of them and rela-
tionships between them studied in [7, 9, 10]. This paper presents a short and updated
description of some part of author’s Lecture on RAAI Summer School 2019. Due to the
limit on the size of the paper it was not possible to include all topics discussed in [9]
and in this lecture. The presented paper can be considered as complementary to the [9].
It includes new methods of construction of correlation functions presented recently on
INES 2019 [10] and contains more examples of (dis)similarity and correlation func-
tions illustrating these methods. The list of references includes only several works.
Hundreds or thousands papers have been published on related topics during more than
one hundred years. Of course it was not possible to include most of them. Some useful
references can be found in the papers cited in the list of references.
2 Examples of Similarity Measures and Correlation

Coefficients for Different Domains
2.1 Binary n-Tuples

Consider n-tuple x ¼ ðx1 ; . . .; xn Þ, xi 2 f0; 1g; i ¼ 1; . . .; n. It can represent an object
x described by n binary features, attributes or properties such that xi ¼ 1 if the object
x possesses the i-th attribute and xi ¼ 0 in the opposite case. Denote X the set of
attributes possessed by x. In statistics, a binary n-tuple can denote n measurements of a
Data Science: Similarity, Dissimilarity and Correlation Functions 15
dichotomous variable or property x for n sampling units. xi ¼ 1 denotes that the

property x fulfilled for the unit i, and xi ¼ 0 otherwise. We use here the first inter-
pretation of binary n-tuples. For two binary n-tuples x ¼ ðx1 ; . . .; xn Þ; y ¼ ðy1 ; . . .; yn Þ
denote:
• a the number of attributes possessed by both objects x and y, when xi ¼ yi ¼ 1;
• b the number of attributes possessed by x and not by y, when xi ¼ 1; yi ¼ 0;
• c the number of attributes possessed by y and not by x, when xi ¼ 0; yi ¼ 1;
• d the number of attributes not possessed by both objects x and y, when xi ¼ yi ¼ 0:
The numbers a and d usually referred to as the numbers of positive and negative
matches, respectively. Below are examples of similarity and association measures for
binary n-tuples [13]:
Jaccard similarity measure:
a jX \ Y j
SJ ¼ ¼ ;
a þ b þ c jX [ Y j
Simple Matching similarity measure:
aþd \ Y j
jX \ Y j þ jX
SSM ¼ ¼ ;
aþbþcþd n
Hamann coefficient:
\ Y j jX \ Y j jX
ða þ d Þ ðb þ cÞ jX \ Y j þ jX \ Yj
AH ¼ ¼ ;
aþbþcþd n
Yule’s Q association coefficient:
\ Y j jX \ Y j jX
ad bc jX \ Y j jX \ Yj
AYQ ¼ ¼ \ Yj :
\ Y j þ jX \ Y j jX
ad þ bc jX \ Y j jX
2.2 Real Valued n-Tuples

Consider n-tuples x ¼ ðx1 ; . . .; xn Þ; y ¼ ðy1 ; . . .; yn Þ with real valued components.
Cosine similarity measure:
Pn
xi yi
cosðx; yÞ ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Pn i¼1 2
ffi pP ffi;
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n 2
i¼1 xi i¼1 yi
where xi ; yi 0, i ¼ 1; . . .; n.
16 I. Z. Batyrshin
Pearson’s product-moment correlation coefficient:

Pn
ðxi xÞðyi yÞ
r ¼ P i¼1
q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ;
ffi qP
n 2 n
i¼1 ðxi xÞ i¼1 ðyi yÞ 2
P P
where x ¼ 1n ni¼1 xi , y ¼ 1n ni¼1 yi :
For both measures, it is supposed that denominators do not equal to zero.
2.3 Rankings
Consider two rankings of n objects x ¼ ðx1 ; . . .; xn Þ; y ¼ ðy1 ; . . .; yn Þ containing n dif-
ferent integer ranks, 1 xi ; yi n, i.e. “without ties”.
Spearman’s rank correlation coefficient [12, 18, 22]:
P
6 ni¼1 di2
q¼1 ;
nð n2 1Þ
where di ¼ xi yi ; i ¼ 1; . . .; n.
Kendall’s rank correlation coefficient [12, 18, 22] for real valued n-tuples x ¼
ðx1 ; . . .; xn Þ; y ¼ ðy1 ; . . .; yn Þ without ties, i.e. xi ¼
6 xj , yi 6¼ yj for all i; j:
NC ND
s¼ ;
nðn 1Þ=2
where NC is the number of concordant pairs ði; jÞ calculated as follows:

Xn1 Xn
1; if xi xj yi yj [ 0
NC ¼ s ;
j¼i þ 1 ij
and sij ¼ ;
i¼1 0; otherwise
and ND is the number of disconcordant pairs calculated as follows:

Xn1 Xn
1; if xi xj yi yj \0
ND ¼ j¼i þ 1
dij ; and dij ¼ :
i¼1 0; otherwise
Note that only the ordering of the component values of n-tuples is used.
2.4 Finite Probabilistic Distributions

Suppose x ¼ ðx1 ; . . .; xn Þ is a finite probabilistic distribution, i.e. xi 0, for all
P
n
i ¼ 1; . . .; n, and xi ¼ 1.
i¼1
Bhattacharyya coefficient is defined as follows [1]:

Xn pffiffiffiffiffiffiffi
Sðx; yÞ ¼ i¼1
xi yi :
2.5 Kendall’s General Correlation Coefficient

Kendall [22] considered the problem of constructing general correlation coefficient.
For two n-tuples x ¼ ðx1 ; . . .; xn Þ and y ¼ ðy1 ; . . .; yn Þ he defined it as:
Pn
i;j¼1 aij bij
r ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Pn Pn ;
2 2
i;j¼1 aij i;j¼1 bij
where the values aij calculated from the values xi and xj , similarly bij calculated from yi
and yj , such that the following conditions are fulfilled: aij ¼ aji , bij ¼ bji , aii ¼
bii ¼ 0 for all i; j ¼ 1; . . .; n. As particular cases of this formula he obtained Spearman’s
and Kendall’s rank correlation coefficients, and Pearson’s product-moment correlation
coefficient. How to obtain other correlation coefficients from this formula does not
clear.
3 Similarity and Dissimilarity Functions
The examples of similarity measures and correlation coefficients considered in the

previous section show that there is a variety of such indicators for different types of
data. Generally, there are introduced tens of such measures and coefficients [13, 14,
27]. To analyze the general properties of these measures, possible relationships
between them and to study the methods of their construction in [9] it was proposed to
consider these measures as functions defined on a universal domain X and satisfying
some simple sets of properties. Three main types of such functions have been con-
sidered: similarity, dissimilarity and correlation functions, which can be used as models
of similarity, dissimilarity and correlation measures and coefficients, respectively.
Below we introduce these functions and consider some of their properties. More results
can be found in [9].
Further, for brevity, similarity and dissimilarity functions will be referred to as (dis)
similarity functions or as resemblance functions.
3.1 Definition of Similarity, Dissimilarity and Correlation Functions

Let X be a nonempty set that will be referred to as a universal domain or an underlying
set in definition of similarity and correlation functions. As X we can consider domain
specific for a considered data type: the set of all binary n-tuples, the set of all real
valued vectors of the length n, the set of all fuzzy sets defined on some domain X, the
set of some images or objects considered in some problem, etc.
18 I. Z. Batyrshin
A function S : X X ! ½0; 1 is called a similarity function on X if for all x, y in X

it is symmetric:
Sðx; yÞ ¼ Sðy; xÞ;
and reflexive:
Sðx; xÞ ¼ 1:
A function D : X X ! ½0; 1 is a dissimilarity function on X if for all x, y in X it

is symmetric:
Dðx; yÞ ¼ Dðy; xÞ;
and irreflexive:
Dðx; xÞ ¼ 0:
These functions are called complementary if for all x, y in X it is fulfilled:
Sðx; yÞ þ Dðx; yÞ ¼ 1:
For complementary (dis)similarity functions we have:
Sðx; yÞ ¼ 1 Dðx; yÞ; Dðx; yÞ ¼ 1 Sðx; yÞ: ð1Þ
A function A : X X ! ½1; 1 is a correlation function (association measure) on

X if for all x, y in X it is symmetric:
Aðx; yÞ ¼ Aðy; xÞ;
reflexive:
Aðx; xÞ ¼ 1;
and negative: Aðx; yÞ\0 for some x, y in X.

Such correlation functions will be referred to as weak correlation functions if they
will not satisfy inverse relationship property considered in Sect. 4. In Sect. 4 we define
a strong (invertible) correlation function on a set X with involution operation.
It is easy to see that the indicators considered in the previous section belong to the
following classes of functions:
• Similarity functions: Jaccard, Simple Matching, Cosine (if all xi and yi have non-
negative values) similarity measures and Bhattacharyya coefficient;
• Correlation functions: Hamann and Yule’s Q coefficients, Pearson’s product-
moment correlation coefficient, Spearman’s and Kendall’s rank correlation coeffi-
cients, Cosine (if xi and yi can have positive and negative values).
3.2 Examples of Complementary (Dis)Similarity Functions

Jaccard (see Sect. 2.1):
a jX \ Y j
SJ ðx; yÞ ¼ ¼ ;
bþc jX Y j
DJ ðx; yÞ ¼ ¼ :
Simple Matching (Sect. 2.1):
aþd \ Y j
jX \ Y j þ jX
SSM ðx; yÞ ¼ ¼ ;
aþbþcþd n
bþc jX Y j 1 Xn
DSM ðx; yÞ ¼ ¼ ¼ jx yi j:
1 i
aþbþcþd n n
Cosine (Sect. 2.2):

Pn
i¼1 xi yi
cosðx; yÞ ¼ pP
n 2
ffi pP ffi ; xi ; yi 0; i ¼ 1; . . .; n:
n 2
i¼1 xi i¼1 yi
!2
1 Xn xi yi
Dðx; yÞ ¼ Pn 2ffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Pn 2ffi :
2 i¼1
i¼1 xi i¼1 yi
Bhattacharyya (Sect. 2.4):

Xn pffiffiffiffiffiffiffi
Sðx; yÞ ¼ i¼1
xi yi :
1 Xn pffiffiffiffi pffiffiffiffi2
Dðx; yÞ ¼ x i yi :
2 i¼1
The last dissimilarity function is called Hellinger discrimination, Matusita measure

or Squared-Chord distance.
3.3 Fuzzy Relations and Kleene Algebra of Resemblance Functions

Here, similarity and dissimilarity functions will be also referred to as resemblance
functions. A resemblance function is a symmetric function R : X X ! ½0; 1, which
is reflexive or irreflexive. We will say that two resemblance functions have the same
type if both are reflexive or both are irreflexive.
Resemblance function R can be considered as a fuzzy relation [2, 17, 28, 29], given
by membership function: lR : X X ! ½0; 1, where lR ðx; yÞ is the strength of the
relation R between x and y. The properties of fuzzy relations can be extended on
resemblance functions.
20 I. Z. Batyrshin
Denote P ðSÞ; P ðDÞ and P ðRÞ the sets of all similarity, dissimilarity and resem-
blance functions, respectively, defined on the set X. We have P ðSÞ; P ðDÞ P ðRÞ and
P ðSÞ [ P ðDÞ ¼ P ðRÞ. Define the operations intersection \ and union [ of resem-
blance functions R1 and R2 on the set X for all x, y in X as follows:
ðR1 \ R2 Þðx; yÞ ¼ minfR1 ðx; yÞ; R2 ðx; yÞg;
ðR1 [ R2 Þðx; yÞ ¼ maxfR1 ðx; yÞ; R2 ðx; yÞg:
The set P ðRÞ will be a distributive lattice [11], partially ordered by the relation:
R1 R2 if R1 ðx; yÞ R2 ðx; yÞ for all x; y in X.
The sets P ðSÞ and P ðDÞ are distributive sublattices of P ðRÞ. For all similarity
functions S1 ; S2 2 P ðSÞ and dissimilarity functions D1 ; D2 2 P ðDÞ it is fulfilled:
S1 \ S2 ; S1 [ S2 2 P ðSÞ; D1 \ D2 ; D1 [ D2 2 P ðDÞ;
S1 \ D1 2 P ðDÞ; S1 [ D1 2 P ðSÞ:
Consider similarity and dissimilarity functions defined for all x, y in X by:

0; if x ¼ y
D0 ðx; yÞ ¼ 0; D1 ðx; yÞ ¼ :
1; otherwise

1; if x ¼ y
S0 ðx; yÞ ¼ ; S1 ðx; yÞ ¼ 1;
0; otherwise
D0 is the least element of P ðDÞ and P ðRÞ, S1 is the greatest element of P ðSÞ and
P ðRÞ, S0 is the least element of P ðSÞ and D1 is the greatest elements of P ðDÞ.
The complement N ðRÞ of a resemblance function R is defined for all x, y in X by:
N ðRÞðx; yÞ ¼ 1 Rðx; yÞ. This operation is involutive, i.e. for all R in P ðRÞ we have:
N ðN ðRÞÞ ¼ R. The complement of the similarity and dissimilarity functions will be
equal to their complementary dissimilarity and similarity functions (1), respectively:
N ðDÞ ¼ S; N ðSÞ ¼ D:
The lattice P ðRÞ with the complement N will be a normal De Morgan (Kleene)
algebra [5] where for any resemblance functions R1 and R2 De Morgan laws:
N ðR1 \ R2 Þ ¼ N ðR1 Þ [ N ðR2 Þ; N ðR1 [ R2 Þ ¼ N ðR1 Þ \ N ðR2 Þ;
and normality:
R1 \ N ðR1 Þ R2 [ N ðR2 Þ;
are fulfilled.
Another random document with
no related content on Scribd:
CHAPTER I
INTRODUCTION
Cincinnati, the city of my birth and early memories, was, in the

‘sixties, about as begrimed and noisy and altogether unattractive as
any place well could be; yet it possessed certain attributes which
really entitled it to the proud designation of “The Queen City of the
West.”
It was prosperous; it had hardly yet been surpassed in prosperity
by Chicago; Cleveland was not even spoken of as a rival; and in many
ways it was the most important centre west of New York and east of
the Mississippi.
It owed its early development principally to its advantageous
location. It lay on the great central route from the East to the West,
which runs from Baltimore and Washington to Cumberland and over
the Alleghenies to Pittsburg, thence by the Ohio River to Cincinnati
and on west to St. Louis and south to New Orleans. It had an
important trade with New Orleans and drew commerce from a large
territory to the north. But whatever else may be said of it, its most
devoted citizen could not claim that Cincinnati was beautiful. Its
buildings were unlovely; its streets were badly paved and as badly
kept; and it lay under a pall of soft coal smoke which left its sooty
mark upon everything—inhabitants included.
Yet, ugly as it was, the city boasted an unusual society. During the
first half of the nineteenth century many young men of good stock
and great ability, drawn by the promise of rapid advancement, had
moved to Cincinnati from all parts of the East and South; New
Jersey, New England, Virginia and Kentucky contributing, perhaps,
the greatest number. There were many families of wealth and culture
which, without parade or display, maintained fine homes and
dispensed a generous hospitality. The suburbs, East Walnut Hills,
Mt. Auburn and Clifton, on the heights to the north and east, were
famous for their beautiful country places.
Then there was a large population of the best class of Germans,
many of whom were university men who left their own country after
the Revolution of 1848 and came to Cincinnati to settle. Of these,
Frederick Hassaurek, General Willich and Judge Stallo, who came to
Cincinnati when Carl Schurz went to St. Louis, are perhaps the most
prominent. The German influence upon the community was marked.
It made for a more liberal Sunday; it brought the study of German
into the public schools; and it developed a strong taste for good
music. Indeed, the musical advantages of Cincinnati in my girlhood
were better than those of any city in the United States, with the
exception of New York or Boston. Theodore Thomas was president of
the Conservatory of Music and he organised a symphony orchestra
which he continued to direct until he went to Chicago along about
1890.
Cincinnati in those days, with her educated, wealthy and public-
spirited society, was much in advance of any other city in the
Mississippi Valley in culture and refinement. There was great
interest in schools of all sorts and in every kind of intellectual
activity. Away back in 1848 the Literary Club of Cincinnati was
formed by a company of men among whom were both Mr. Taft’s
father and mine, as well as Rutherford B. Hayes, Stanley Matthews,
Manning F. Force and Mr. Spofford, later Librarian of Congress. This
club continues to be a cherished institution and in my girlhood it was
the centre of all interest in literature and intellectual pursuits.
My father, John Williamson Herron, was a graduate of Miami
University at Oxford, Ohio, and was in college with Benjamin
Harrison. He was for fifty years a trustee of that institution and was
devoted to its interests. My husband’s father, Judge Alphonso Taft,
was one of the Yale class of 1833, was for many years a member of
the Yale Corporation, and had five sons who graduated at that
university. My mother’s brother, Judge Isaac Clinton Collins, and
one of my two brothers also graduated at Yale, while my other
brother graduated at Harvard, so it will be seen that both my
husband and I grew up in the midst of strong collegiate traditions.
To write about one’s childhood is not easy. Memories by the score
come flocking up, but, dear as they are, upon examination they turn
out to be quite commonplace and hardly worth relating. My
memories are not sufficiently “early” to have any special value. The
first thing that I dimly remember is sitting on the front steps of my
home watching some sort of parade in which there were many
soldiers, but I was too young then to know that it was a peace
celebration I was witnessing at the close of the Civil War.
My father was a lawyer who came to the bar of Ohio in the ’forties.
He was United States Attorney under President Harrison, was a State
Senator, and twice declined appointments to the Bench because the
salary attached to these positions was not enough to support his
large family. I was the fourth in a family of eleven, eight girls and
three boys. One boy and two girls died before I can remember.
Our house was one of a block of grey brick houses in Pike Street, at
the east end of Cincinnati, which, at that time, was the fashionable
residence section of the city. Pike Street runs down to the river on a
rather steep incline and, as it was paved with cobblestones, my early
memories are somewhat marred by an impression of the frequent
clatter and clang of heavy wagons pulling their way up the hill from
the river landing.
While our house was not particularly distinguished, being much
like those on either side of it, across the street from us there were two
very striking and imposing residences which lent distinction to the
neighbourhood, and in which, as I grew up, were formed the
pleasantest associations of my life. The one directly opposite was a
large, square, red brick house which had an air of great dignity. It
was the home of Mr. Larz Anderson. There were ten boys in the
Anderson family and, though they were all much older than I and
most of them had gone away before I grew up, I remember that it
was a very lively household always. In my later girlhood we were
specially linked to this family by the marriage of one of the boys,
Charles, to my sister Jennie.
The house next to Mr. Anderson’s, on the north, I knew as the
Sinton home. A low, colonial structure, well set in a garden of green
lawns and finely kept shrubbery, it is still one of the most beautiful
residences in Cincinnati, and, indeed, in the whole country. Its
architecture suggests that of the White House and it was, as a matter
of fact, designed by the same architect, an Irishman named Hoban.
The Sinton house is lower than the White House, being only one
story high with a basement, but it has the same classic outlines and it
bears, moreover, the stamp of time, which gives it a character all its
own.
It was built about 1800 by a Mr. Martin Baum, but was purchased
by the first Nicholas Longworth in the early part of the century and
was the home of the Longworth family for a generation. Long before
I can remember, it was bought by Mr. David Sinton, one of the most
successful business men in Ohio, and to me it was always the Sinton
home. When I was about twelve years old, Mr. Sinton’s daughter
Annie married my husband’s brother, Charles P. Taft, and as they
have always lived in this old house it has come to be known, since
Mr. Sinton’s death in 1901, as the Taft house. It is the only Taft house
in Cincinnati now, the house where my husband was born having
been sold after his father’s death, and it has been the scene of many
of the most important events of my life. It was there that my husband
received the announcement of his nomination for the Presidency; it
was there, in front of the house, that he made his speech of
acceptance; and it was there that Charles Taft gathered a large party
of friends on the night of November 6, 1908, to receive with us the
election returns. And it is now to this house, where my husband’s
brother Charles and his wife dispense a generous hospitality, that we
always go when we return to Cincinnati.
MRS. TAFT’S CHILDHOOD HOME ON
PIKE STREET, IN CINCINNATI
My girlhood days were spent quite placidly in Miss Nourse’s

school, which was known in Cincinnati as “The Nursery,” and where
all the girls of the Herron family, as well as Mr. Taft’s only sister,
Fanny, received their education. Miss Nourse was a Maine woman
with a thorough New England education and with a thoroughly New
England idea of imparting it. She insisted, especially, upon languages
and literature. Much of my time, outside of that taken up in regular
school work, I devoted to the study of music, and I practised my
scales on the family piano with such persistence that I wonder the
whole neighbourhood did not rebel. Music was the absorbing interest
of my life in those days, the inspiration of all my dreams and
ambitions.
Our house was none too large for the family, but as there was a
wide difference in our ages it happened that my oldest sister was
married while my youngest sister was still a baby in long clothes.
Then, the boys went away to college and were gone the better part of
each year, so it was not often that we were all at home together.
Nevertheless, we had our share of the happy-go-lucky and somewhat
crowded existence of a large family on a moderate income.
My mother was Harriet Collins, and when she was seventeen years
old she came with her mother to Cincinnati, from Lowville, New
York, to live with her brother, Judge Collins, who was my father’s law
partner and continued to be so for more than forty years. Her father,
Eli Collins, was a Member of Congress from the Lowville district of
New York. My mother was in many ways a remarkable, as well as a
most attractive, woman. She had an exceedingly keen wit and a mind
alert to the humour in every situation. With so many children to
nurse, to scold, to sew for and, sometimes, to cook for—in a word, to
bring up on a small income—she would seem to have had little time
for outside interests; but she was very popular in society and I
remember that in her busiest years she went out a great deal. She
had a stimulating personality and I do know that she made her
family circle a very amusing and interesting one in which to grow up.
The only incident of my girlhood which was in any way unusual
was my first visit to the White House as a guest of President and Mrs.
Hayes. Mr. and Mrs. Hayes and my father and mother had been
lifelong friends. Mr. Hayes was, at one time, a partner in my father’s
law firm. They had been closely associated for a great many years
and had a very warm regard for each other. My youngest sister was
born shortly after the election of Mr. Hayes, was named Lucy Hayes
Herron, after Mrs. Hayes, and was taken to the White House to be
christened. My mother paid several visits to the White House and
after my sister Jennie was married Mrs. Hayes invited her and Mr.
Anderson to stay a week with her and, to my intense excitement, she
added that she would like to have me accompany them. I was
seventeen years old; I had never been to Washington and to me it
was a very important event. I was not “out,” so I couldn’t spend my
time in the White House as I would have liked, in going to brilliant
parties and meeting all manner of charming people, but, fortunately
for my peace of mind, the Hayes lived very quietly, so it was not so
trying to have to devote myself to seeing the sights of the Capital like
any other tourist.
I didn’t meet my husband until I was eighteen years old. We had
been born and brought up in the same town; our fathers were warm
friends and had practised law at the same bar for more than forty
years; during that time our mothers had exchanged visits, and my
sister Maria and Fanny Taft were schoolmates and close companions
at Miss Nourse’s, but the Tafts lived at Mt. Auburn, a hill suburb of
Cincinnati, and after Will finished Woodward High School he went
for four years to Yale, so it is not at all surprising that we did not
meet.
Judge Alphonso Taft was Secretary of War, and later Attorney
General, in Grant’s Cabinet while his son Will was at college, but
before the latter graduated, the family had returned to Cincinnati, so
he came straight home and entered at once upon a law course in the
Cincinnati Law School. It was at that time, when he was still a
student and working as a law reporter on the Cincinnati
Commercial, that I met him. It was at a coasting party one winter’s
night, I remember very well, when I went with a party of young
people, including the Charles Tafts, to coast down a fine steep hill in
Mt. Auburn. Will Taft was there, and after being introduced to me he
took me down the hill on his big bobsled. After that we met very
frequently.
A small circle of us went in for amateur theatricals with much
enthusiasm and great earnestness. We launched ourselves in our
histrionic careers in “She Stoops to Conquer” which we gave at the
house of one of the company. Then came “A Scrap of Paper” in Mrs.
Charles Taft’s drawing room, in which both Will and I took part. We
had become very ambitious by this time and sent all the way to New
York for a professional stage-manager to help us with the
production. But it turned out a most nervous occasion. We were all
overtrained, I suppose. One thing after another went wrong until at
the crisis of the play, where the hero is supposed to find in the barrel
of a gun the scrap of paper upon which the whole plot hinges, the
amateur hero looked pretty foolish when he discovered there wasn’t
any gun. Another one of the company, in a fit of absentmindedness,
no doubt due to overwrought nerves, had carried it off the stage, and
just when the situation was getting tragic for the hero the culprit
came creeping back with it and carefully put it where it belonged, for
all the world as if he thought he were making himself invisible to the
audience.
But our ardour was not dampened. I remember Mr. Taft especially
in a burlesque of “The Sleeping Beauty,” which, in its legitimate
form, had been produced for charity at Pike’s Opera House. The
Unity Club, a most respectable organization of the young men of the
Unitarian Church, decided to give their version of the same story,
and it was a huge success. Mr. Taft played the title rôle and his
brother Horace, who is six feet four in his stocking feet, shared with
the Beauty the honours of the evening as a most enchanting Puck.
Then we had parties in the country, too. Many of our friends had
country places that spread along the Madison Road and the Grandin
Road on East Walnut Hills, and two of my closest friends lived out
there in a great house, looking down over the majestic but tawny
Ohio River, above the point where the sweeping curve begins that
carries it by the amphitheatre in which the business part of the city is
built. It was a long distance to East Walnut Hills and in my girlhood
we had to go the greater part of the way in a clumsy old omnibus that
clumped along over the unpaved roads at the rate of about three
miles an hour. But such little inconveniences didn’t trouble us, and
many were the vaudeville and charade parties that we had, there
being enough “talent” among us to get up an amusing performance at
a moment’s notice.
MR. AND MRS. JOHN WILLIAMSON HERRON, MRS. TAFT’S
FATHER AND MOTHER
But in spite of all this gaiety, Mr. Taft was making very satisfactory
progress in his career. As a law reporter he showed his growing
interest in the public welfare by meeting certain elements in
Cincinnati politics with vigorous denunciation. There was a man
named Tom Campbell, a clever criminal lawyer, who had something
more than a suspicion against him of bribery and corruption of both
witnesses and juries, and he had succeeded in organising a political
machine that was running the town according to his directions.
Campbell was counsel for the defence in what was known as the
Hoffman case and was strongly suspected of tampering with the jury,
and Mr. Taft in reporting the case, took special pains to bring out all
the fine points in the lawyer’s character and methods, telling the
truth as he saw it.
This brought him into association with Mr. Miller Outcalt, the
Assistant Prosecuting Attorney, who represented the State in the
Hoffman case, and when Mr. Outcalt succeeded by election to the
position of prosecuting attorney he offered the place of assistant to
Mr. Taft, although he had been at the bar not more than seven
months. Mr. Taft served in this office for fourteen months and the
experience he had in the rough-and-ready practice in criminal trials,
in preparing cases for trial, in examining witnesses, in making
arguments to the court and in summing up to the jury, was the most
valuable experience he could possibly have in fitting him for trial
work at the bar.
But this experience was shortened by a circumstance not of his
seeking. Major Benjamin Butterworth was the Congressman from
one of the Cincinnati districts in President Arthur’s administration,
and the President being anxious to relieve the Collector of Internal
Revenue, called on Major Butterworth to suggest the name of
another man. Major Butter worth had been for a long time a warm
friend of Mr. Taft, thought he had a good family name and was too
young in politics to have many political enemies, so he suggested him
and wrote to urge him to accept the appointment which the
President immediately offered to him. He accepted the place and
held it for a year, but it proved a serious interruption in his legal
career. He resigned as soon as it was possible and began practice
with Major H. P. Lloyd who had been his father’s partner before he
went to Vienna.
Mr. Taft went abroad in the summer of 1883 to visit Judge and
Mrs. Taft in Vienna, and it was about this time, when we had all
spent several years in frivolities, that several of us became very
serious-minded and decided that we must have something by way of
occupation more satisfying than dancing and amateur theatricals. I
secured a position as school teacher and taught for two years, first at
Madame Fredin’s and then at White and Sykes, both private schools
out on Walnut Hills. Then, with two of my intimate friends, I decided
to start a “salon.” We called it a “salon” because we planned to
receive a company who were to engage in what we considered
brilliant discussion of topics intellectual and economic, and we
decided that our gathering should include only specially invited
guests. Among these were the two Taft brothers, Will and Horace,
and other men common friends of us all.
In view of the fact that two marriages resulted from this salon, Mr.
Taft has suggested ulterior motives on the part of those who got it up,
but there was no truth in the charge. We were simply bent on
“improving our minds” in the most congenial atmosphere we could
create, and if our discussions at the salon usually turned upon
subjects of immediate personal interest, to the neglect of the abstruse
topics we had selected for debate, it was because those subjects were
just then claiming the attention of the whole community.
Cincinnati, thanks to the activities of Tom Campbell and his
followers, was then in a tangle of political mismanagement of a
particularly vicious character, and our little circle developed a civic
spirit which kept us alive to local interests to the exclusion, for the
time being, of everything else. Mr. Taft was intimately connected
with the reform movement, and in all its phases, through comedy
and tragedy, disappointment and elation, we fought it out at our
salon meetings with such high feeling and enthusiasm that its history
became the history of our lives during that period.
Then came the famous Berner case. This was in 1884. Berner had
committed a deliberate murder of an unusually appalling nature and
with robbery as the motive, and there was great excitement about it.
Campbell became his counsel and, in a trial which held the attention
of the community while it lasted, he succeeded in getting the man off
for manslaughter when the unanimous opinion was that he should
have been hanged. Nobody could see how an honest jury could have
rendered any other verdict. There was intense indignation
throughout the city and a meeting was called to denounce Campbell
as an embracer of juries and a suborner of perjury.
On the evening when the meeting to denounce Campbell was
called we were having a session of the salon and our whole
discussion was of the possible developments which might grow out of
the infamous Berner trial. We were greatly excited about it. I
remember the evening distinctly because of the terrible things that
happened. We were disturbed by a great commotion in the street and
we sallied forth in a body to see what it was all about.
The mass meeting was held at Music Hall and was presided over
by Dr. Kemper, a very effective speaker. The crowd was angry and
quickly passed the condemnatory resolutions which were framed.
But with all the indignation and resentment everything might have
been carried out quite calmly had not the match been applied to the
powder. Just as the meeting was breaking up somebody shouted:
“Let’s go down to the jail and take Berner out!”
It was an appeal to the mob spirit which responds so readily in an
angry crowd; they went; and of course the worst elements
immediately came to the top. They attacked the jail, which was in the
rear of the court house, but were held back until the militia, which
had been instantly summoned, arrived. Then they went around to
the front and set fire to the court house. With the streets packed with
raging humanity it was not possible to fight the fire and the building
was completely destroyed.
The militia charged the mob and this inspired somebody with the
idea of raiding a gun store and seizing arms and ammunition with
which to make a resistance. The idea caught on and spread rapidly.
One place attacked was Powell’s gun shop near Fourth and Main. But
Powell, either forewarned or foreseeing some such development, had
quietly made preparations to meet it. He lighted up the front of the
store as brightly as he could, then, with two or three other men who
were expert shots, he put himself behind a barricade in the rear. The
mob came on and as the ringleaders broke into the shop they were
picked off by the men behind the barricade and killed in their tracks.
Four or five of them went down in a heap and the crowd behind
them, not expecting such a reception, instantly was brought to its
senses. This was in April, 1884.
Such an outbreak was a disgrace to the city of Cincinnati, but it
had the effect of bringing the Campbell controversy to a head. A bar
committee of ten men, of which both my father and Mr. Taft were
members, was formed to see what could be done to rid the
community of the evil reputation it had acquired. This committee
made a thorough investigation of Campbell’s character and record,
prepared charges against him and, with my father as chairman,
presented them, in June, 1884, to the district court of three judges,
and asked a hearing and Campbell’s disbarment if the charges were
proved.
MEMBERS OF THE SALON. MR. TAFT IN THE CENTER, WITH
THE AUTHOR AT HIS RIGHT
Campbell had been indicted on a criminal charge of attempting to

bribe a man called on the Berner jury and the prosecutor in this case
was our intimate friend and associate, Mr. Rufus Smith, who had
been in Europe with Mr. Taft the year before. The jury hung, eight to
four, although the evidence was strong against the defendant. This
fanned the flames of popular resentment and I don’t suppose our
little salon was the only place in Cincinnati where Campbell was
carefully retried and convicted. In this criminal case Mr. Foraker,
who shortly afterward became Governor of Ohio, was counsel for
Campbell.
The disbarment hearing was set for the following November and
some six months was thus given for taking the depositions of non-
resident witnesses. Mr. Kittredge and Mr. Ramsey, leaders of the bar,
were retained as senior counsel for the committee, and Mr. Taft and
Mr. John Holmes, a warm friend of ours, were junior counsel and
were directed to prepare the evidence. In this work Mr. Taft and Mr.
Holmes went all over the country taking depositions and we kept in
constant touch with them. All the members of the committee
expected to have their reputations assailed, being perfectly certain
that Campbell would not hesitate at any measure he might be able to
take to discredit them, but they went ahead nevertheless.
When the trial came on Mr. Ramsey, of the senior counsel,
expected to open the case, but he became quite seriously ill and was
confined to his house for days. Through his unexpected absence, the
duty of making the opening statement fell to Mr. Taft. He was taken
completely by surprise, but he rose to the opportunity, which was
certainly a splendid one for a man so young. He had then been at the
bar only four years, but having assisted throughout in the
preparation of the evidence he knew the case from beginning to end
and he made a speech which lasted four hours and a half. Mr. Taft
thinks this was an opportunity improved which had an important
influence on his career. The special part it played in his subsequent
promotion I shall speak of.
The result in the Campbell case was at first disappointing because
the Court which heard the disbarment charges found Campbell guilty
only on minor charges and, by a vote of two to one acquitted him on
those which would have required his disbarment. But the public
disapproval of the Court’s decision and the moral effect of the
proceedings drove Campbell from the city and the State and
accomplished the purpose of the bar association.
The Campbell trial was finished in December, 1884, and in
January, 1885, Mr. Rufus Smith, an old and intimate friend, entered
the office of County Solicitor and tendered to Mr. Taft the place of
Assistant County Solicitor. The advantage of this office was that it
paid $2500 a year and that, while he acted as counsel for the county,
he still was able to continue the general practice of law with his
partner, Major Lloyd.
Mr. Taft and I were engaged in May, 1885, and were married in
June of the following year.
In the summer of 1885 my mother, moved I think by some
sentimental attachment to the scenes of her childhood, decided that
she would take us all up into the Adirondacks, to a little camp near
Lowville. My two older sisters were married so there were only six of
us left in the family, but we were still something of a handful to move
in a body. However, my mother was equal to it. We packed almost a
van load of trunks and set out, and one evening we arrived, over the
worst corduroy road that was ever laid down, at a little cottage beside
a beautiful lake in a setting of pine-clad hills. The scenery indeed was
most satisfactory, but the cottage was so small that the family more
than strained its capacity. Then we took our meals at a sort of
boarding house called Fenton’s, where the only thing on the bill of
fare was fresh beef. I like what is known as “roughing it” as well as
anybody, but even the superlative appetite produced by outdoor
living demands some variety; and variety we did not get.
Mr. Taft had elected to remain in Cincinnati all summer and save
money. It was a Spartan resolution and we all applauded it, but he
probably found Lowville a long way from Pike Street; and I certainly
thought Mother was sacrificing a good deal for the sake of renewing
the memories of her youth. However, the days went on, while the
fresh beef grew less and less tempting.
I had written Mr. Taft something about the Fenton fare and he,
wanting very much to join us, but having no excuse for breaking his
admirable resolution to remain in Cincinnati, hit upon the only plan
for escaping comment on his lack of fortitude. He went down to
Peeble’s, a fancy grocer, and selecting a box as big as a Saratoga
trunk, ordered it filled with every kind of delicacy he could think of
or have pointed out to him and brought it with him to Lowville.
We went rowing on the lake about sundown the evening he
arrived, and right in the middle of a fine long stroke he suddenly
dropped his oars, reached in his pocket and drew out a letter. He
laughed a little when he handed it to me, then picking up his oars he
rowed on without a word. The letter was from his father.
Judge Taft was at this time Minister to St. Petersburg, having been
transferred from Vienna. Will had written him about his engagement
and about his plan to remain in town all summer and devote himself
strictly to business and the accumulation of funds; and this was the
answer.
There were a lot of nice complimentary things about me, with the
warmest congratulations and good wishes; then the letter closed by
saying: “I am very much pleased with your decision to remain in
Cincinnati this summer. I myself have found it not at all bad if you
take care of yourself, and there is no doubt that during the quiet
months one can make and save considerable money by staying at
home. I congratulate you on your strength of character.” We really
had a delightful summer at Fenton’s after that.
My father had given me a very nice lot at the end of McMillan
Street on the site of an old quarry, which commanded a fine view of
the Ohio River and the surrounding country, and Mr. Taft and I
determined to build a house on it which should be ready for us when
we got back from our wedding trip. So the winter before our
marriage was filled with architects’ plans, contractors’ estimates and
all the other fascinating details of building, and we thought that we
had finally settled upon a design that met with every requirement of
good taste and modern comfort.
For our wedding trip, we went abroad, and I had my first taste of
the foreign travel of which I had always dreamed. We crossed on the
City of Chester which was the oldest, and therefore the cheapest ship
of the Inman line. We chose her for the simple reason that her rates
accorded with our means, but we found, much to our astonishment,
that we were the only people on board who had deliberately selected
her. Everybody else had been forced to take her because of some
emergency or some mishap. One man had to miss the Germanic in
order to give his dentist time to relieve a very troublesome tooth.
Another man was called to court just as he was about to board the
Britannic. Those were the proud ships of the Atlantic in those days
and it was not at all difficult to understand why anybody should
prefer them to the City of Chester, but it amused us greatly to hear
the shamefaced excuses of our fellow passengers. My husband and I
were not ashamed, nor were we so particular about our comforts that
we did not thoroughly enjoy ourselves. Besides, we had the gratifying
consciousness of the money which the low rates had left in our
pockets to be spent much more profitably abroad.
MR. AND MRS. WILLIAM HOWARD TAFT AT THE TIME OF
THEIR MARRIAGE
The trip was full of interest to us both. We spent the greater part of
the summer in England and saw the sights of London and the
cathedral towns in great detail. Our only trip on the Continent was
through Holland to Paris. I remember that in Amsterdam I bought
some old and rather large Delft plates. They wouldn’t go into any
trunk we had, so I had them carefully packed in a wicker hamper and
this article became thereafter a part of our hand luggage, and was the
occasion for a decided disagreement between my husband and me as
to what the true object of travel was. He used to say that he “toted
that blamed thing all around Europe and after all it arrived in
Cincinnati with its contents in small pieces.” Which was true. He had
“toted” it all around Europe, but when we arrived in New York I
entrusted it to an express company with the result that when we
opened it we found its contents in such a condition that only an
accomplished porcelain mender could put a sufficient number of
pieces together to make what my husband always afterward referred
to as “the memento of our first unpleasantness.”
Our trip from Cincinnati to Cincinnati took just one hundred days
and cost us just one thousand dollars, or five dollars a day each. I
venture to say that could not be done nowadays, even by as prudent a
pair as we were.
During a subsequent trip abroad, two years later, I was able to
indulge my desire to hear music. We went to Beyreuth, to the
Wagner festival, and heard Parsifal and The Meistersingers
gloriously rendered; after which we went to Munich and attended
operas and concerts until Mr. Taft rebelled. He said that he enjoyed a
certain amount of music just as much as anybody, but that he did
want to get something more out of European travel than a nightly
opera and a daily symphony.
So—we went to Italy and saw Rome and Florence in true
Baedecker style. When we arrived in Rome we opened our Baedecker
and read that there was almost no foundation for Rome’s awful
reputation as an unhealthy place. “Rome is a very healthy place,” said
Baedecker, “at all times of the year except the first two weeks in
August, when a visit there is attended with risk.” We had arrived for
the first two weeks in August!
When we came home from our wedding trip we found that our
house was not yet completed, so we went to stay with Judge and Mrs.
Taft for a month at the old house in Mt. Auburn. It was a nice old
place, with about three acres of ground, but the air around it was just
about as sooty as if it had been located down under the factory
chimneys. Mt. Auburn is on a sort of promontory which juts out into
the city; it is on a level with the tops of the smoke stacks and it
catches all the soot that the air can carry that far.
Judge and Mrs. Taft had come home from their European mission
in time for our wedding. Judge Taft had been ill in St. Petersburg and
had given his family a great deal of anxiety, but he was now settled
down to the business of quiet recuperation and the enjoyment of
well-earned rest.
My husband’s father was “gentle” beyond anything I ever knew. He
was a man of tremendous firmness of purpose and just as set in his
views as any one well could be, but he was one of the most lovable
men that ever lived because he had a wide tolerance and a strangely
“understanding sympathy” for everybody. He had a great many
friends, and to know him was to know why this was so.
Mr. Taft’s mother, though more formal, was also very kindly and
made my visit to her home as a bride full of pleasure. The two, the
father and mother, had created a family atmosphere in which the
children breathed in the highest ideals, and were stimulated to
sustained and strenuous intellectual and moral effort in order to
conform to the family standard. There was marked serenity in the
circle of which Judge and Mrs. Taft were the heads. They had an
abiding confidence in the future of their children which strongly
influenced the latter to justify it. They both had strong minds,
intellectual tastes, wide culture and catholic sympathies.
Not long after we arrived my husband came to me one day with an
air of great seriousness, not to say of conciliation and said:
“Nellie, Father has got himself into rather a difficulty and I hope I
can rely on you to help him out—not make it too hard for him, you
know,—make him feel as comfortable about it as you can. The truth
is he used to have a messenger at the War Department in
Washington whom he was very fond of. He was a bright man—
colored, of course—and he was very devoted to Father. Now this man
called on Father down town to-day. He’s here on a private car and
Father says he’s made a great success as a porter. Father got to
talking to him, and there were lots of things they wanted to talk
about, and besides the man said he would like very much to see
Mother,—and Father, who was just about ready to come home to
lunch said—right on the spur of the moment—you understand he
didn’t think anything about it—he said to this man, ‘Come on home
and have lunch with us.’ He’s downstairs now. Father came to me
and said he had just realised that it was something of a difficulty and
that he was sorry. He said that he could take care of Mother if I could
take care of you. So I hope you won’t mind.”
As soon as I could control my merriment caused by this halting
and very careful explanation, I went down to luncheon. I didn’t mind
and Will’s mother didn’t mind, but the expression on the face of
Jackson, the negro butler, was almost too much for my gravity. I will
say that the porter had excellent manners and the luncheon passed
off without excitement.
We made a short visit at my mother’s on Pike Street before we
moved into our new house on McMillan Street; but we began the
year of 1887 under our own roof which, though it was mortgaged,
was to us, for the time being, most satisfactory.

PDF Artificial Intelligence 5Th Raai Summer School Dolgoprudny Russia July 4 7 2019 Tutorial Lectures Gennady S Osipov Ebook Full Chapter

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PDF Artificial Intelligence 5Th Raai Summer School Dolgoprudny Russia July 4 7 2019 Tutorial Lectures Gennady S Osipov Ebook Full Chapter

Uploaded by

Copyright:

Available Formats

Artificial Intelligence 5th RAAI Summer

School Dolgoprudny Russia July 4 7

Business Intelligence 5th European Summer School eBISS

Business Intelligence Third European Summer School

Reasoning Web Declarative Artificial Intelligence 16th

Business Intelligence 4th European Summer School eBISS

Engineering Trustworthy Software Systems: 5th

Artificial Intelligence XXXVI 39th SGAI International

Advances in Neural Computation Machine Learning and

Lectures on the Geometry of Tensors Informal Notes for

Subseries of Lecture Notes in Computer Science

Konstantin S. Yakovlev (Eds.)

ISSN 0302-9743 ISSN 1611-3349 (electronic)

LNCS Sublibrary: SL7 – Artiﬁcial Intelligence

© Springer Nature Switzerland AG 2019

July 2019 Gennady S. Osipov

Andrey Filchenkov ITMO University, Russia

Hybrid Intelligent Systems Based on Fuzzy Logic and Deep Learning . . . . . . 3

Data Science: Similarity, Dissimilarity and Correlation Functions . . . . . . . . . 13

Mathematical Foundation of Cognitive Computing Based

A Review of Motivational Systems and Emotions in Cognitive

Selected Challenges in Grammar-Based Text Generation

Multi-Agent Path Finding – An Overview . . . . . . . . . . . . . . . . . . . . . . . . . 96

Young Scientist School Papers

The Use of Reinforcement Learning in the Task of Moving Objects

Ontology Models in Intelligent System Engineering: A Case

Automated Acquisition, Representation and Processing of Temporal

Natural Language Processing with DeepPavlov Library and Additional

Toward Faster Reinforcement Learning for Robotics:

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Federal Research Centre of Informatics and Computer Science of RAS, Moscow,

Abstract. The purpose of this lecture is to establish the fundamental links

Keywords: Deep learning Neural networks Rule extraction Convolutional

2 Methods for Extracting Rules from the Neural Network

3 Extracting Rules from Deep Neural Networks

Picture 1. Alexnet - ﬁrst deep learning champion in image recognition

Picture 2. Dataset Imagenet.

To semantic intepretation of the deep learning blackbox neuro-fuzzy networks can

Fig. 3. Structure of ANFIS model for time series forecasting

Instituto Politécnico Nacional, Centro de Investigación en Computación,

Abstract. The lecture presents a new, non-statistical approach to the analysis

Keywords: Similarity measure Pearson’s product-moment correlation

© Springer Nature Switzerland AG 2019

2 Examples of Similarity Measures and Correlation

2.1 Binary n-Tuples

dichotomous variable or property x for n sampling units. xi ¼ 1 denotes that the

Simple Matching similarity measure:

Yule’s Q association coefﬁcient:

2.2 Real Valued n-Tuples

Pearson’s product-moment correlation coefﬁcient:

where NC is the number of concordant pairs ði; jÞ calculated as follows:

and ND is the number of disconcordant pairs calculated as follows:

2.4 Finite Probabilistic Distributions

Bhattacharyya coefﬁcient is deﬁned as follows [1]:

2.5 Kendall’s General Correlation Coefﬁcient

3 Similarity and Dissimilarity Functions

The examples of similarity measures and correlation coefﬁcients considered in the

3.1 Deﬁnition of Similarity, Dissimilarity and Correlation Functions

A function S : X  X ! ½0; 1 is called a similarity function on X if for all x, y in X

Sðx; yÞ ¼ Sðy; xÞ;

A function D : X  X ! ½0; 1 is a dissimilarity function on X if for all x, y in X it

Dðx; yÞ ¼ Dðy; xÞ;

These functions are called complementary if for all x, y in X it is fulﬁlled:

A function S : X X ! ½0; 1 is called a similarity function on X if for all x, y in X

A function D : X X ! ½0; 1 is a dissimilarity function on X if for all x, y in X it

A function A : X X ! ½1; 1 is a correlation function (association measure) on

R1 R2 if R1 ðx; yÞ R2 ðx; yÞ for all x; y in X.