Textbook Grammatical Inference Algorithms Routines and Applications 1St Edition Wojciech Wieczorek Auth Ebook All Chapter PDF

Grammatical Inference: Algorithms,
Routines and Applications 1st Edition

Wojciech Wieczorek (Auth.)
Visit to download the full and correct content document:
https://textbookfull.com/product/grammatical-inference-algorithms-routines-and-applic
ations-1st-edition-wojciech-wieczorek-auth/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Biota Grow 2C gather 2C cook Loucas
https://textbookfull.com/product/biota-grow-2c-gather-2c-cook-
loucas/
Computer Age Statistical Inference Algorithms Evidence

and Data Science 1st Edition Bradley Efron
https://textbookfull.com/product/computer-age-statistical-
inference-algorithms-evidence-and-data-science-1st-edition-
bradley-efron/
Computer Age Statistical Inference Algorithms Evidence

and Data Science 1st Edition Bradley Efron
https://textbookfull.com/product/computer-age-statistical-
inference-algorithms-evidence-and-data-science-1st-edition-
bradley-efron-2/
Inference for Heavy Tailed Data Applications in

Insurance and Finance 1st Edition Liang Peng
https://textbookfull.com/product/inference-for-heavy-tailed-data-
applications-in-insurance-and-finance-1st-edition-liang-peng/
Universal Chess Training 1st Edition Wojciech Moranda
https://textbookfull.com/product/universal-chess-training-1st-
edition-wojciech-moranda/
Grouping Genetic Algorithms: Advances and Applications

1st Edition Michael Mutingi
https://textbookfull.com/product/grouping-genetic-algorithms-
advances-and-applications-1st-edition-michael-mutingi/
Swarm Intelligence Algorithms: Modifications and

Applications 1st Edition Adam Slowik
https://textbookfull.com/product/swarm-intelligence-algorithms-
modifications-and-applications-1st-edition-adam-slowik/
Data Mining Algorithms in C++: Data Patterns and

Algorithms for Modern Applications 1st Edition Timothy
Masters
https://textbookfull.com/product/data-mining-algorithms-in-c-
data-patterns-and-algorithms-for-modern-applications-1st-edition-
timothy-masters/
Wireless Algorithms Systems and Applications Sriram

Chellappan
https://textbookfull.com/product/wireless-algorithms-systems-and-
applications-sriram-chellappan/
Studies in Computational Intelligence 673
Wojciech Wieczorek
Grammatical
Inference
Algorithms, Routines and Applications
Studies in Computational Intelligence
Volume 673
Series editor
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: kacprzyk@ibspan.waw.pl
About this Series
The series “Studies in Computational Intelligence” (SCI) publishes new develop-

ments and advances in the various areas of computational intelligence—quickly and
with a high quality. The intent is to cover the theory, applications, and design
methods of computational intelligence, as embedded in the fields of engineering,
computer science, physics and life sciences, as well as the methodologies behind
them. The series contains monographs, lecture notes and edited volumes in
computational intelligence spanning the areas of neural networks, connectionist
systems, genetic algorithms, evolutionary computation, artificial intelligence,
cellular automata, self-organizing systems, soft computing, fuzzy systems, and
hybrid intelligent systems. Of particular value to both the contributors and the
readership are the short publication timeframe and the worldwide distribution,
which enable both wide and rapid dissemination of research output.
More information about this series at http://www.springer.com/series/7092

Wojciech Wieczorek
Grammatical Inference
Algorithms, Routines and Applications
123
Wojciech Wieczorek
Institute of Computer Science
University of Silesia Faculty of Computer
Science and Materials Science
Sosnowiec
Poland
ISSN 1860-949X ISSN 1860-9503 (electronic)

Studies in Computational Intelligence
ISBN 978-3-319-46800-6 ISBN 978-3-319-46801-3 (eBook)
DOI 10.1007/978-3-319-46801-3
Library of Congress Control Number: 2016952872
© Springer International Publishing AG 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made.
Printed on acid-free paper
This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
Grammatical inference, the main topic of this book, is a scientific area that lies at
the intersection of multiple fields. Researchers from computational linguistics,
pattern recognition, machine learning, computational biology, formal learning
theory, and many others have their own contribution. Therefore, it is not surprising
that the topic has also a few other names such as grammar learning, automata
inference, grammar identification, or grammar induction. To simplify the location
of present contribution, we can divide all books relevant to grammatical inference
into three groups: theoretical, practical, and applicable. In greater part this book is
practical, though one can also find the elements of learning theory, combinatorics
on words, the theory of automata and formal languages, plus some reference to
real-life problems.
The purpose of this book is to present old and modern methods of grammatical
inference from the perspective of practitioners. To this end, the Python program-
ming language has been chosen as the way of presenting all the methods. Included
listings can be directly used by the paste-and-copy manner to other programs, thus
students, academic researchers, and programmers should find this book as the
valuable source of ready recipes and as an inspiration for their further development.
A few issues should be mentioned regarding this book: an inspiration to write it,
a key for the selection of described methods, arguments for selecting Python as an
implementation language, typographical notions, and where the reader can send any
critical remarks about the content of the book (subject–matter, listings etc.).
There is a treasured book entitled “Numerical recipes in C”, in which along with
the description of selected numerical methods, listings in C language are provided.
The reader can copy and paste the fragments of the electronic version of the book in
order to produce executable programs. Such an approach is very useful. We can
find an idea that lies behind a method and immediately put it into practice. It is a
guiding principle that accompanied writing the present book.
For the selection of methods, we try to keep balance between importance and
complexity. It means that we introduced concepts and algorithms which are
essential to the GI practice and theory, but omitted that are too complicated or too
v
vi Preface
long to present them as a ready-to-use code. Thanks to that, the longest program
included in the book is no more than a few pages long.
As far as the implementation language is concerned, the following requirements
had to be taken into account: simplicity, availability, the property of being firmly
established, and allowing the use of wide range of libraries. Python and FSharp
programming languages were good candidates. We decided to choose IronPython
(an implementation of Python) mainly due to its integration with the optimization
modeling language. We use a monospaced (fixed-pitch) font for the listings of
programs, while the main text is written using a proportional font. In listings,
Python keywords are in bold.
The following persons have helped the author in preparing the final version of
this book by giving valuable advice. I would like to thank (in alphabetical order):
Prof. Z.J. Czech (Silesian University of Technology), Dr. P. Juszczuk, Ph.D. stu-
dent A. Nowakowski, Dr. R. Skinderowicz, and Ph.D. student L. Strak (University
of Silesia).
Sosnowiec, Poland Wojciech Wieczorek

2016
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 The Problem and Its Various Formulations . . . . . . . . . . . . . . . . . . 1
1.1.1 Mathematical Versus Computer Science Perspectives . . . . . 1
1.1.2 Different Kinds of Output . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.3 Representing Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.4 Complexity Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Assessing Algorithms’ Performance . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 Measuring Classifier Performance . . . . . . . . . . . . . . . . . . . . 7
1.2.2 McNemar’s Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.3 5 2 Cross-Validated Paired t Test . . . . . . . . . . . . . . . . . . 9
1.3 Exemplary Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 Peg Solitaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.2 Classification of Proteins . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Bibliographical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2 State Merging Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Evidence Driven State Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3 Gold’s Idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Grammatical Inference with MDL Principle . . . . . . . . . . . . . . . . . . 27
2.4.1 The Motivation and Appropriate Measures . . . . . . . . . . . . . 28
2.4.2 The Proposed Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3 Partition-Based Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 The k-tails Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 Grammatical Inference by Genetic Search . . . . . . . . . . . . . . . . . . . 37
3.3.1 What Are Genetic Algorithms? . . . . . . . . . . . . . . . . . . . . . . 37
vii
viii Contents
3.3.2 Basic Notions of the Genetic Algorithm for GI. . . . . . . . . . 37

3.3.3 Our Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4 CFG Inference Using Tabular Representations . . . . . . . . . . . . . . . . 40
3.4.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Substring-Based Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.1 Error-Correcting Grammatical Inference . . . . . . . . . . . . . . . . . . . . . 47
4.1.1 The GI Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2 Alignment-Based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.2.1 Alignment Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.2.2 Selection Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5 Identification Using Mathematical Modeling . . . . . . . . . . . . . . . . . . . . 57
5.1 From DFA Identification to Graph Coloring . . . . . . . . . . . . . . . . . . 57
5.1.1 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.2 From NFA Identification to a Satisfiability Problem . . . . . . . . . . . . 61
5.2.1 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5.3 From CFG Identification to a CSP . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.3.1 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6 A Decomposition-Based Algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.1 Prime and Decomposable Languages . . . . . . . . . . . . . . . . . . . . . . . 69
6.1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.1.2 Cliques and Decompositions . . . . . . . . . . . . . . . . . . . . . . . . 70
6.2 CFG Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.2.1 The GI Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
7 An Algorithm Based on a Directed Acyclic Word Graph . . . . . . . . . 77
7.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
7.2 Constructing a DAWG From a Sample . . . . . . . . . . . . . . . . . . . . . 78
Contents ix
7.3 Our Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
8 Applications of GI Methods in Selected Fields . . . . . . . . . . . . . . . . . . 83
8.1 Discovery of Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.1.1 Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.1.2 The Schützenberger Methodology . . . . . . . . . . . . . . . . . . . . 84
8.1.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
8.2 Minimizing Boolean Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
8.2.1 Background and Terminology . . . . . . . . . . . . . . . . . . . . . . . 94
8.2.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
8.2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
8.3 Use of Induced Star-Free Regular Expressions . . . . . . . . . . . . . . . . 100
8.3.1 Definitions and an Algorithm . . . . . . . . . . . . . . . . . . . . . . . 100
8.3.2 An Application in Classification of Amyloidogenic
Hexapeptides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
8.3.3 An Application in the Construction of Opening Books . . . . 105
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Appendix A: A Quick Introduction to Python . . . . . . . . . . . . . . . . . . . . . 111
Appendix B: Python’s Tools for Automata, Networks, Genetic
Algorithms, and SAT Solving. . . . . . . . . . . . . . . . . . . . . . . . 129
Appendix C: OML and its Usage in IronPython . . . . . . . . . . . . . . . . . . . 139
Acronyms
CFG Context-free grammar

CGT Combinatorial game theory
CNF Chomsky normal form
CNF Conjunctive normal form
CSP Constraint satisfaction problem
DFA Deterministic finite automaton
DNF Disjunctive normal form
EDSM Evidence driven state merging
GA Genetic algorithm
GI Grammatical inference
GNF Greibach normal form
ILP Integer linear programming
LP Linear programming
MDL Minimum description length
MILP Mixed integer linear programming
NFA Non-deterministic finite automaton
NLP Non-linear programming
NP Non-deterministic polynomial time
OGF Ordinary generating function
OML Optimization modeling language
PTA Prefix tree acceptor
RPNI Regular positive and negative inference
SAT Boolean satisfiability problem
TSP Traveling salesman problem
XML Extensible markup language
xi
Chapter 1
Introduction
1.1 The Problem and Its Various Formulations
Let us start with the presentation of how many variants of a grammatical inference
problem we may be faced with. Informally, we are given a sequence of words and
the task is to find a rule that lies behind it. Different models and goals are given
by response to the following questions. Is the sequence finite or infinite? Does the
sequence contain only examples (positive words) or also counter-examples (negative
words)? Is the sequence of the form: all positive and negative words up to a certain
length n? What is meant by the rule: are we satisfied with regular acceptor, context-
free grammar, context-sensitive grammar, or other tool? Among all the rules that
match the input, should the obtained one be of a minimum size?
1.1.1 Mathematical Versus Computer Science Perspectives
The main division of GI models comes from the size of a sequence. When it is infinite,
we deal with mathematical identification in the limit. The setting of this model is that
of on-line, incremental learning. After each new example, the learner (the algorithm)
must return some hypothesis (an automaton or a CFG). Identification is achieved
when the learner returns a correct answer and does not change its decision afterwards.
With respect to this model the following results have been achieved: (a) if we are
given examples and counter-examples of the language to be identified (learning from
informant), and each individual word is sure of appearing, then at some point the
inductive machine will return the correct hypothesis; (b) if we are given only the
examples of the target (learning from text), then identification is impossible for any
super-finite class of languages, i.e., a class containing all finite languages and at least
one infinite language. In this book, however, we only consider the situation when
the input is finite, which can be called a computer science perspective. We are going
to describe algorithms the part of which base on examples only, and the others base
© Springer International Publishing AG 2017 1

W. Wieczorek, Grammatical Inference, Studies in Computational Intelligence 673,
DOI 10.1007/978-3-319-46801-3_1
2 1 Introduction
Fig. 1.1 A DFA accepting a b c

aa∗ bb∗ cc∗
a b c
S A B C
on both examples and counter-examples. Sometimes we will demand the smallest

possible form of an output, but every so often we will be satisfied with an output that
is just consistent with the input. Occasionally, an algorithm gives the collection of
results that gradually represent the degree of generalization of the input.
1.1.2 Different Kinds of Output
The next point that should be made is that how important it is to pinpoint the kind of
a target. Consider the set of examples: {abc, aabbcc, aaabbbccc}. If a solution is
being sought in the class of regular languages, then one possible guess is presented in
Fig. 1.1. This automaton matches every word starting with one or more as, followed
by one or more bs, and followed by one or more cs. If a solution is being sought
in the class of context-free languages, then one of possible answers is the following
grammar:
S → ABC A → a|aAB
B→b C →c|CC
It is clearly seen that the language accepted by this CFG is {am bm cn : m, n ≥ 1}.
Finally, if a solution is being sought in the class of context-sensitive languages, then
even more precise conjecture can be made:
S →aBc a A → aa b A → Ab
B → Ab B c B c → bc
Now, the language accepted by this grammar is {am bm cm : m ≥ 1}. It is worth

emphasizing that from the above-mentioned three various acceptors, the third one
can not be described as a CFG or a DFA, and the second one can not be described as
a DFA (or an NFA). In the book we will only consider the class of regular and context-
free languages. The class of context-sensitive languages is very rarely an object of
investigations in the GI literature. Such identification is thought to be a very hard
task. Moreover, the decision problem that asks whether a certain word belongs to the
language of a given context-sensitive grammar, is PSPACE-complete.
It should be noted that in the absence of counter-examples, we are at risk of
over-generalization. An illustration that suggests itself is the regular expression (a +
b + c)∗ , which represents all words over the alphabet {a, b, c}, as a guess for
the examples from the previous paragraph. In the present book, two approaches
1.1 The Problem and Its Various Formulations 3
to this problem have been proposed. First, the collection of different hypotheses
is suggested, and it is up to a user to select the most promising one. Second, the
minimum description length (MDL) principle is applied, in which not only the size
of an output matters but also the amount of information that is needed to encode
every example.
1.1.3 Representing Languages
Many definitions specific to particular methods are put in relevant sections. Herein,
we give definitions that will help to understand the precise formulation of our GI
problem and its complexity. Naturally, we skip conventional definitions and notation
from set theory, mathematical logic, and discrete structures, in areas that are on
undergraduate courses.
Definition 1.1 Σ will be a finite nonempty set, the alphabet. A word (or sometimes
string) is a finite sequence of symbols chosen from an alphabet. For a word w, we
denote by |w| the length of w. The empty word λ is the word with zero occurrences of
symbols. Sometimes, to be coherent with external notation, we write epsilon instead
of the empty word. Let x and y be words. Then x y denotes the catenation of x and
y, that is, the word formed by making a copy of x and following it by a copy of y.
We denote, as usual by Σ ∗ , the set of all words over Σ and by Σ + the set Σ ∗ − {λ}.
A word w is called a prefix (resp. a suffix) of a word u if there is a word x such that
u = wx (resp. u = xw). The prefix or suffix is proper if x = λ. Let X, Y ⊂ Σ ∗ .
The catenation (or product) of X and Y is the set X Y = {x y | x ∈ X, y ∈ Y }. In
particular, we define

n
X 0 = {λ}, X n+1 = X n X (n ≥ 0), X ≤n = Xi. (1.1)
i=0
Definition 1.2 Given two words u = u 1 u 2 . . . u n and v = v1 v2 . . . vm , u < v accord-

ing to lexicographic order if u = v1 . . . vn (n < m) or if u i < vi for the minimal i
where u i = vi . The quasi-lexicographic order on the Σ ∗ over an ordered alphabet
Σ orders words firstly by length, so that the empty word comes first, and then within
words of fixed length n, by lexicographic order on Σ n .
To simplify the representations for languages (i.e., sets of words), we define the
notion of regular expressions, finite-state automata, and context-free grammars over
alphabet Σ as follows.
Definition 1.3 The set of regular expressions (regexes) over Σ will be the set of
words R such that
1. ∅ ∈ R which represents the empty set.
2. Σ ⊆ R, each element a of the alphabet represents language {a}.
4 1 Introduction
3. If r A and r B are regexes representing languages A and B, respectively, then

(r A + r B ) ∈ R, (r A r B ) ∈ R, (r A∗ ) ∈ R representing A ∪ B, AB, A∗ , respectively,
where the symbols (,), +, ∗ are not in Σ.
We will freely omit unnecessary parentheses from regexes assuming that catena-
tion has higher priority than + and ∗ has higher priority than catenation. If r ∈ R
represents language A, we will write L(r ) = A.
The above definition of regular expressions has a minor drawback. When we
write, for instance, abc, it is unknown whether the word denotes itself or one of the
following regexes: ((ab)c), (a(bc)). In the present book, however, it will be always
clear from the context.
Definition 1.4 A non-deterministic finite automaton (NFA) is defined by a quintuple

(Q, Σ, δ, s, F), where Q is the finite set of states, Σ is the input alphabet, δ : Q ×
Σ → 2 Q is the transition function, s ∈ Q is the initial state, and F ⊆ Q is the set
of final states. When there is no transition at all from a given state on a given input
symbol, the proper value of δ is ∅, the empty set. We extend δ to words over Σ by
the following inductive definition:
1. δ(q, λ) = {q},

2. δ(q, ax) = r ∈δ(q,a) δ(r, x), where x ∈ Σ ∗ , and a ∈ Σ.
Having extended the transition function δ from the domain Q × Σ to the domain
Q × Σ ∗ , we can formally define that a word x belongs to the language accepted by
automaton A, x ∈ L(A), if and only if δ(s, x) ∩ F = ∅.
Definition 1.5 If in a NFA A for every pair (q, a) ∈ Q × Σ, the transition function
holds |δ(q, a)| ≤ 1, then A is called a deterministic finite automaton (DFA).
The transition diagram of an NFA (as well as DFA) is an alternative way to

represent an NFA. For A = (Q, Σ, δ, s, F), the transition diagram of A is a labeled
digraph (V, E) satisfying
a
V = Q, E = {q −→ p : p ∈ δ(q, a)},
a
where q −→ p denotes an edge (q, p) with label a. Usually, final states are depicted
as double circled terms. A word x over Σ is accepted by A if there is a labeled path
from s to a state in F such that this path spells out the word x.
Definition 1.6 A context-free grammar (CFG) is defined by a quadruple G =

(V, Σ, P, S), where V is an alphabet of variables (or sometimes non-terminal sym-
bols), Σ is an alphabet of terminal symbols such that V ∩ Σ = ∅, P is a finite set of
production rules of the form A → α for A ∈ V and α ∈ (V ∪ Σ)∗ , and S is a special
non-terminal symbol called the start symbol. For the sake of simplicity, we will write
A → α1 | α2 | . . . | αk instead of A → α1 , A → α1 , . . . , A → αk . We call a word
x ∈ (V ∪ Σ)∗ a sentential form. Let u, v be two words in (V ∪ Σ)∗ and A ∈ V .
Then, we write u Av ⇒ uxv, if A → x is a rule in P. That is, we can substitute word
1.1 The Problem and Its Various Formulations 5
x for symbol A in a sentential form if A → x is a rule in P. We call this rewriting

a derivation. For any two sentential forms x and y, we write x ⇒∗ y, if there exists
a sequence x = x0 , x1 , . . . , xn = y of sentential forms such that xi ⇒ xi+1 for all
i = 0, 1, . . . , n − 1. The language L(G) generated by G is the set of all words over
Σ that are generated by G; that is, L(G) = {x ∈ Σ ∗ | S ⇒∗ x}.
Definition 1.7 A sample S over Σ will be an ordered pair S = (S+ , S− ) where S+ ,
S− are finite subsets of Σ ∗ and S+ ∩ S− = ∅. S+ will be called the positive part of
S (examples), and S− the negative part of S (counter-examples). Let A be one of
the following acceptors: a regular expression, a DFA, an NFA, or a CFG. We call A
consistent (or compatible) with a sample S = (S+ , S− ) if and only if S+ ⊆ L(A)
and S− ∩ L(A) = ∅.
1.1.4 Complexity Issues
The consecutive theorems are very important in view of computational intractability

of GI in its greater part. These results will be given without proofs because of the
scope and nature of the present book. Readers who are interested in details of this
may get familiar with works listed at the end of this chapter (Sect. 1.4).
Theorem 1.1 Let Σ be an alphabet, S be a sample over Σ, and k be a positive
integer. Determining whether there is a k-state DFA consistent with S is NP-complete.
It should be noted, however, that the problem can be solved in polynomial time if
S = Σ ≤n for some n. The easy task is also finding such a DFA A that L(A) = S+
and A has the minimum number of states.
Theorem 1.2 The decision version of the problem of converting a DFA to a minimal
NFA is PSPACE-complete.
Let us answer the question of whether NFA induction is a harder problem than
DFA induction. The search space for the automata induction problem can be assessed
by the number of automata with a fixed number of states. It has been shown that the
number of pairwise non-isomorphic minimal k-state DFAs over a c-letter alphabet is
of order k2k−1 k (c−1)k , while the number of NFAs on k states over a c-letter alphabet
2
such that every state is reachable from the start state is of order 2ck . Thus, switching
from determinism to non-determinism increases the search space enormously. On the
other hand, for c, k ≥ 2, there are at least 2k−2 distinct languages L ⊆ Σ ∗ such that:
(a) L can be accepted by an NFA with k states; and (b) the minimal DFA accepting L
has 2k states. It is difficult to resist the conclusion that—despite its hardness—NFA
induction is extremely important and deserves exhaustive research.
Theorem 1.3 Let Σ be an alphabet, S be a sample over Σ, and k be a positive
integer. Determining whether there is a regular expression r over Σ, that has k
or fewer occurrences of symbols from Σ, and such that r is consistent with S, is
NP-complete.
6 1 Introduction
Interestingly, the problem remains NP-complete even if r is required to be star-free

(contains no “∗” operations).
When moving up from the realm of regular languages to the realm of context-free
languages we are faced with additional difficulties. The construction of a minimal
cover-grammar seems to be intractable, specially in view of the following facts: (a)
there is no polynomial-time algorithm for obtaining the smallest context-free gram-
mar that generates exactly one given word (unless P = NP); (b) context-free gram-
mar equivalence and even equivalence between a context-free grammar and a regular
expression are undecidable; (c) testing equivalence of context-free grammars gener-
ating finite sets needs exponential time; (d) the grammar can be exponentially smaller
than any word in the language.
1.1.5 Summary
Let Σ be an alphabet, and k, n be positive integers. We will denote by I (input)

a sample S = (S+ , S− ) over Σ for which either S+ , S− = ∅ or S+ = ∅ and S− = ∅.
We write O (output) for one of the following acceptors: a regex, a DFA, an NFA, or
a CFG. The notation |O| = k means that, respectively, a regex has k occurrences of
symbols from Σ, a DFA (or NFA) has k states, and a CFG has k variables.
From now on, every GI task considered in the book will be put into one of the
three formulations:
1. For a given I find a consistent O.
2. For given I and k > 0 find a consistent O such that |O| ≤ k.
3. For a given I find a consistent O such that |O| is minimal.
But not all combinations will be taken into account. For example, in the book there
is no algorithm for finding a minimum regex or for finding a CFG for examples only.
For technical or algorithmic reasons, sometimes we will demand S not including λ.
If we are given a huge I with positive and negative words along with an algorithm
A that works for positive words only,1 then the following incremental procedure
can be applied. First, sort S+ in a quasi-lexicographic order to (s1 , s2 , . . . , sn ). Next,
set j = 2 (or a higher value), infer A (I = {s1 , . . . , s j−1 }), and check whether
L(A (I )) ∩ S− = ∅. If no counter-example is accepted, find the smallest k from the
indexes { j, j + 1, . . . , n} for which sk is not accepted by A (I ), and set j = k + 1.
Take the set I = I ∪ {sk } as an input for the next inference. Pursue this incremental
procedure until j = n + 1. Note that the proposed technique is a heuristic way of
reducing the running time of the inference process, with no guarantee of getting the
correct solution.
technique can also be used for algorithms that work on S = (S+ , S− ), provided that their
1 This
computational complexity primarily depends on the size of S+ .

1.2 Assessing Algorithms’ Performance 7
1.2 Assessing Algorithms’ Performance
Suppose that we are given a sample S from a certain domain. How would we evaluate
a GI method or compare more such methods. First, a proper measure should be
chosen. Naturally, it depends on a domain’s characteristic. Sometimes only precision
would be sufficient, but in other cases relying only on a single specific measure—
without calculating any general measure of the quality of binary classifications (like
Matthews correlation coefficient or others)—would be misleading.
After selecting the measure of error or quality, we have to choose between three
basic scenarios. (1) The target language is known, and in case of regular languages
we simply check the equivalence between minimal DFAs; for context-free languages
we are forced to generate the first n words in the quasi-lexicographic order from
the hypothesis and from the target, then check the equality of two sets. Random
sampling is also an option for verifying whether two grammars describe the same
language. When the target is unknown we may: (2) random split S into two subsets,
the training and the test set (T&T) or (3) apply K -fold cross-validation (CV), and then
use a selected statistical test. Statisticians encourage us to choose T&T if |S| > 1 000
and CV otherwise. In this section, McNemar’s test for a scenario (2) and 5 × 2 CV
t test for a scenario (3), are proposed.
1.2.1 Measuring Classifier Performance
Selecting one measure that will describe some phenomenon in a population is cru-
cial. In the context of an average salary, take for example the arithmetic mean and
the median from the sample: $55,000, $59,000, $68,000, $88,000, $89,000, and
$3,120,000. Our mean is $579833.33. But it seems that the median,2 $78,000 is
better suited for estimating our data. Such examples can be multiplied. When we
are going to undertake an examination of binary classification efficiency for selected
real biological or medical data, making the mistake to classify a positive object as
a negative one, may cause more damage than incorrect putting an object to the group
of positives; take examples—the ill, counter-examples—the healthy, for good illus-
tration. Then, recall (also called sensitivity, or a true positive rate) would be more
suited measure than accuracy, as recall quantifies the avoiding of false negatives.
By binary classification we mean mapping a word to one out of two classes by
means of inferred context-free grammars and finite-state automata. The acceptance
of a word by a grammar (resp. an automaton) means that the word is thought to
belong to the positive class. If a word, in turn, is not accepted by a grammar (resp. an
automaton), then it is thought to belong to the negative class. For binary classification
a variety of measures has been proposed. Most of them base on rates that are shown
2 The median is the number separating the higher half of a data sample, a population, or a probability
distribution, from the lower half. If there is an even number of observations, then there is no single
middle value; the median is then usually defined to be the mean of the two middle values.
8 1 Introduction
Table 1.1 Confusion matrix for two classes

True class
Predicted class Positive Negative Total
Positive tp: true positive fp: false positive p
Negative fn: false negative tn: true negative n
Total p n N
in Table 1.1. There are four possible cases. For a positive object (an example), if
the prediction is also positive, this is a true positive; if the prediction is negative for
a positive object, this is a false negative. For a negative object, if the prediction is
also negative, we have a true negative, and we have a false positive if we predict
a negative object as positive. Different measures appropriate in particular settings
are given below:
• accuracy, ACC = (tp + tn)/N ,

• error, ERR = 1 − ACC,
• balanced accuracy, BAR = (tp/ p + tn/n)/2,
• balanced error, BER = 1 − BAR,
• precision, P = tp/p ,
• true positive rate (eqv. with recall, sensitivity), TPR = tp/p,
• specificity, SPC = tn/n,
• F-measure, F1 = 2 × P × TPR/(P + TPR), √
• Matthews correlation coefficient, MCC = (tp × tn − fp × fn)/ p × p × n × n .
If the class distribution is not uniform among the classes in a sample (e.g. the set of
examples forms a minority class), so as to avoid inflated measurement, the balanced
accuracy should be applied. It has been shown that BAR is equivalent to the AUC (the
area under the ROC curve) score in case of the binary (with 0/1 classes) classification
task. So BAR is considered as primary choice if a sample is highly imbalanced.
1.2.2 McNemar’s Test
Given a training set and a test set, we use two algorithms to infer two acceptors
(classifiers) on the training set and test them on the test set and compute their errors.
The following natural numbers have to be determined:
• e1 : number of words misclassified by 1 but not 2,
• e2 : number of words misclassified by 2 but not 1.
Under the null hypothesis that the algorithms have the same error rate, we expect
e1 = e2 . We have the chi-square statistic with one degree of freedom
1.2 Assessing Algorithms’ Performance 9
(|e1 − e2 | − 1)2
∼ χ12 (1.2)
e1 + e2
and McNemar’s test rejects the hypothesis that the two algorithms have the same
error rate at significance level α if this value is greater than χα,1
2
. For α = 0.05,
χ0.05,1 = 3.84.
2
1.2.3 5 × 2 Cross-Validated Paired t Test
This test uses training and test sets of equal size. We divide the dataset S randomly
into two parts, x1(1) and x1(2) , which gives our first pair of training and test sets. Then
we swap the role of the two halves and get the second pair: x1(2) for training and x1(1)
( j)
for testing. This is the first fold; xi denotes the j-th half of the i-th fold. To get the
second fold, we shuffle S randomly and divide this new fold into two, x2(1) and x2(2) .
We then swap these two halves to get another pair. We do this for three more folds.
( j)
Let pi be the difference between the error rates (for the test set) of the two
classifiers (obtained from the training set) on fold j = 1, 2 of replication i =
1, . . . , 5. The average on replication i is pi = ( pi(1) + pi(2) )/2, and the estimated
variance is si2 = ( pi(1) − pi )2 + ( pi(2) − pi )2 . The null hypothesis states that the two
algorithms have the same error rate. We have t statistic with five degrees of freedom
p (1)
1 ∼ t5 . (1.3)
5
i=1 is 2
/5
The 5×2 CV paired t test rejects the hypothesis that the two algorithms have the same
error rate at significance level α if this value is outside the interval (−tα/2, 5 , tα/2, 5 ).
If significance level equals 0.05, then t0.025, 5 = 2.57.
1.3 Exemplary Applications
As a concept, grammatical inference is a broad field that covers a wide range of

possible applications. We can find scientific research and practical implementations
from such fields as:
• Natural language processing: building syntactic parsers, language modeling, mor-
phology, phonology, etc. In particular, one of its founding goals is modeling lan-
guage acquisition.
• Bioinformatics and biological sequence analysis: modeling automatically RNA
and protein families of sequences.
10 1 Introduction
• Structural pattern recognition is a field in which grammatical inference has been

active since 1974.
• Program engineering and software testing: verification, testing, specification
learning.
• Music: classification, help in creating, data recovering.
• Others: malware detection, document classification, wrapper induction (XML),
navigation path analysis, and robotic planning.
Some bibliographical references associated with these topics have been reported
in Sect. 1.4. Further in this section, a selected CGT (combinatorial game theory)
problem as well as a problem from the domain of bioinformatics are solved as
an illustration of GI methods usage.
1.3.1 Peg Solitaire
The following problem has been formulated in one of CGT books:

Find all words that can be reduced to one peg in one-dimensional Peg Solitaire.
(A move is for a peg to jump over an adjacent peg into an empty adjacent space,
and remove the jumped-over peg: for instance, 1101 → 0011 → 0100, where 1
represents a peg and 0 an empty space.) Examples of words that can be reduced
to one peg are 1, 11, 1101, 110101, 1(10)k 1. Georg Gunther, Bert Hartnell and
Richard Nowakowski found that for an n × 1 board with one empty space, n
must be even and the space must be next but one to the end. If the board is
cyclic, the condition is simply n even.
Let us solve this problem using a GI method. Firstly, it should be noticed that leading
and trailing 0 s are unimportant, so we will only consider words over {0, 1} that
start and end with 1s. Secondly, for every word of length n ≥ 3 going outside, i.e.,
11dd · · · 1 → 100dd · · · 1, will always put us to a position that cannot be reduced
to one peg. Thus, among all moves that can be made from a word w (|w| ≥ 3),
we do not need to take into account those moves which increase the size of a word
(one-dimensional board).
Under the hypothesis that the solution being sought, say H , is a regular language,
the idea for solving the problem can be stated in three steps: (1) for some n generate
H ∩{0, 1}≤n ; (2) using the k-tails method (from Sect. 3.2) find a sequence of automata;
(3) see if any one of them have passed a verification test.
Step 1
To the previously given remarks we can add another one: H is subset of L(1(1 +
01 + 001)∗ ). In order to determine all words from H up to a certain length, dynamic
programming has been applied, using the fact that for any word with m 1s, a jump
leads to a word with m − 1 1s and the length of the word does not increase (for words
consisting of three or more digits). The algorithm based on dynamic programming
will work if as input we give I = L(1(1 + 01 + 001)∗ ) ∩ {0, 1}≤n sorted by the
1.3 Exemplary Applications 11
number of 1 s in ascending order. We can easily find out that |I | = 1104 for n = 12
and |I | = 6872 for n = 15. This leads us to the following algorithm.
from FAdo.fa import *
from FAdo.reex import *
def moves(w):
result = set()
n = len(w)
if w[:3] == ’110’:
result.add(’1’ + w[3:])
for i in xrange(2, n-2):
if w[i-2:i+1] == ’011’:
result.add(w[:i-2] + ’100’ + w[i+1:])
if w[i:i+3] == ’110’:
result.add(w[:i] + ’001’ + w[i+3:])
if w[-3:] == ’011’:
result.add(w[:-3] + ’1’)
return result
def pegnum(w):
c = 0
for i in xrange(len(w)):
if w[i] == ’1’:
c += 1
return c
def generateExamples(n):
"""Generates all peg words of length <= n
Input: n in {12, 15}
Output: the set of examples"""
rexp = str2regexp("1(1 + 01 + 001)*")
raut = rexp.toNFA()
g = EnumNFA(raut)
numWords = {12: 1104, 15: 6872}[n]
g.enum(numWords)
words = sorted(g.Words, \
cmp = lambda x, y: cmp(pegnum(x), pegnum(y)))
S_plus = {’1’, ’11’}
for i in xrange(4, numWords):
if moves(words[i]) & S_plus:
S_plus.add(words[i])
return S_plus
Step 2
The second step is just the invocation of the k-tails algorithm. Because the algorithm
outputs the sequence of hypothesis, its invocation has been put into the for-loop
structure as we can see in a listing from step 3.
Step 3
The target language is unknown, that is why we have to propose any test for probable
correctness of obtaining automata. To this end, we generate two sets of words, namely
positive test set (Test_pos) and negative test set (Test_neg). The former contains
all words form the set H ∩ {0, 1}≤15 , the latter contains the remaining words over
{0, 1} up to the length 15. An automaton is supposed to be correct if it accepts all
words from the positive test set and accepts no word from the negative test set.
12 1 Introduction
0 1
0 1 1
1 1 0 1
1 1 0
0 0 1
1
1
1 0
0 1
1 0
1
Fig. 1.2 A DFA representing H
def allWords(n):
"""Generates all words over {0, 1} up to length n
Input: an integer n
Output: all w in (0 + 1)* such that 1 <= |w| <= n"""
rexp = str2regexp("(0 + 1)(0 + 1)*")
raut = rexp.toNFA()
g = EnumNFA(raut)
g.enum(2**(n+1) - 2)
return set(g.Words)
Train_pos = generateExamples(12)
Test_pos = generateExamples(15)
Test_neg = allWords(15) - Test_pos
for A in synthesize(Train_pos):
if all(A.evalWordP(w) for w in Test_pos) \
and not any(A.evalWordP(w) for w in Test_neg):
Amin = A.toDFA().minimalHopcroft()
print Amin.Initial, Amin.Final, Amin.delta
break
The resulting automaton is depicted in Fig. 1.2.
1.3.2 Classification of Proteins
One of the problems studied in bioinformatics is classification of amyloidogenic

hexapeptides. Amyloids are proteins capable of forming fibrils instead of the func-
tional structure of a protein, and are responsible for a group of diseases called amyloi-
dosis, such as Alzheimer’s, Huntington’s disease, and type II diabetes. Furthermore,
it is believed that short segments of proteins, like hexapeptides consisting of 6-residue
fragments, can be responsible for amyloidogenic properties. Since it is not possible
to experimentally test all such sequences, several computational tools for predicting
amyloid chains have emerged, inter alia, based on physico-chemical properties or
using machine learning approach.
Suppose that we are given two sets, training and test, of hexapeptides:
Train_pos = \
[’STQIIE’, ’STVIIL’, ’SDVIIE’, ’STVIFE’, ’STVIIS’, ’STVFIE’, \
’STVIIN’, ’WIVIFF’, ’YLNWYQ’, ’SFQIYA’, ’SFFFIQ’, ’STFIIE’, \
’GTFFIN’, ’ETVIIE’, ’SEVIIE’, ’YTVIIE’, ’STVIIV’, ’SMVLFS’, \
’STVIYE’, ’VILLIS’, ’SQFYIT’, ’SVVIIE’, ’STVIII’, ’HLVYIM’, \

’IEMIFV’, ’FYLLYY’, ’FESNFN’, ’TTVIIE’, ’STVIIF’, ’STVIIQ’, \
’IFDFIQ’, ’RQVLIF’, ’ITVIIE’, ’KIVKWD’, ’LTVIIE’, ’WVFWIG’, \
’SLVIIE’, ’STVTIE’, ’STVIIE’, ’GTFNII’, ’VSFEIV’, ’GEWTYD’, \
’KLLIYE’, ’SGVIIE’, ’STVNIE’, ’GVNYFL’, ’STLIIE’, ’GTVLFM’, \
’AGVNYF’, ’KVQIIN’, ’GTVIIE’, ’WTVIIE’, ’STNIIE’, ’AQFIIS’, \
’SSVIIE’, ’KDWSFY’, ’STVIIW’, ’SMVIIE’, ’ALEEYT’, ’HYFNIF’, \
’SFLIFL’, ’STVIIA’, ’DCVNIT’, ’NHVTLS’, ’EGVLYV’, ’VEALYL’, \
’LAVLFL’, ’STSIIE’, ’STEIIE’, ’STVIIY’, ’LYQLEN’, ’SAVIIE’, \
’VQIVYK’, ’SIVIIE’, ’HGWLIM’, ’STVYIE’, ’QLENYC’, ’MIENIQ’]
Train_neg = \
[’KTVIVE’, ’FHPSDI’, ’FSKDWS’, ’STVITE’, ’STVDIE’, ’FMFFII’, \
’YLEIII’, ’STVIDE’, ’RMFNII’, ’ETWFFG’, ’NGKSNF’, ’KECLIN’, \
’STVQIE’, ’IQVYSR’, ’AAELRN’, ’EYLKIA’, ’KSNFLN’, ’DECFFF’, \
’STVPIE’, ’YVSGFH’, ’EALYLV’, ’HIFIIM’, ’RVNHVT’, ’AEVLAL’, \
’PSDIEV’, ’STVIPE’, ’DILTYT’, ’RETWFF’, ’STVIVE’, ’KTVIYE’, \
’KLLEIA’, ’QPKIVK’, ’EECLFL’, ’QLQLNI’, ’IQRTPK’, ’YAELIV’, \
’KAFIIQ’, ’GFFYTP’, ’HPAENG’, ’KTVIIT’, ’AARRFF’, ’STVIGE’, \
’LSFSKD’, ’NIVLIM’, ’RLVFID’, ’STVSIE’, ’LSQPKI’, ’RGFFYT’, \
’YQLENY’, ’QFNLQF’, ’ECFFFE’, ’SDLSFS’, ’KVEHSD’, ’STVMIE’, \
’QAQNQW’, ’SSNNFG’, ’TFWEIS’, ’VTLSQP’, ’STVIEE’, ’TLKNYI’, \
’LRQIIE’, ’STGIIE’, ’YTFTIS’, ’SLYQLE’, ’DADLYL’, ’SHLVEA’, \
’SRHPAE’, ’KWDRDM’, ’FFYTPK’, ’STVIQE’, ’GMFNIQ’, ’HKALFW’, \
’LLWNNQ’, ’GSHLVE’, ’VTQEFW’, ’NIQYQF’, ’STMIIE’, ’PTEKDE’, \
’TNELYM’, ’LIAGFN’, ’HAFLII’, ’YYTEFT’, ’EKNLYL’, ’KTVLIE’, \
’FTPTEK’, ’STPIIE’, ’STVVIE’, ’SGFHPS’, ’LFGNID’, ’SPVIIE’, \
’STVISE’, ’EKDEYA’, ’RVAFFE’, ’FYTPKT’, ’PKIQVY’, ’DDSLFF’, \
’ERGFFY’, ’PTVIIE’, ’DIEVDL’, ’STIIIE’]
Test_pos = \
[’FTVIIE’, ’HQLIIM’, ’ISFLIF’, ’GTFFIT’, ’YYQNYQ’, ’HFVWIA’, \
’NTVIIE’, ’SNVIIE’, ’MLVLFV’, ’YVEYIG’, ’STVWIE’, ’STVIIM’, \
’EYSNFS’, ’SQVIIE’, ’SYVIIE’, ’FLVHSS’, ’NQQNQY’, ’QYFNQM’, \
’DTVIIE’, ’VTSTFS’, ’STVIIT’, ’LIFLIV’, ’SFVIIE’, ’NYVWIV’, \
’NFGAIL’, ’STVIID’, ’VTVIIE’, ’MTVIIE’, ’STVLIE’, ’LLYYTE’, \
’QTVIIE’, ’KLFIIQ’, ’ATVIIE’, ’LVEALY’, ’TYVEYI’, ’RVFNIM’, \
’NQFIIS’, ’STVEIE’]
Test_neg = \
[’STDIIE’, ’LKNGER’, ’KAILFL’, ’NYFAIR’, ’VKWDRD’, ’KENIIF’, \
’WVENYP’, ’WYFYIQ’, ’VAQLNN’, ’DLLKNG’, ’HLVEAL’, ’TAWYAE’, \
’STAIIE’, ’STVGIE’, ’ERIEKV’, ’EVDLLK’, ’STVIIP’, ’AINKIQ’, \
’STTIIE’, ’TYQIIR’, ’MYFFIF’, ’TEFTPT’, ’NGERIE’, ’AENGKS’, \
’ICSLYQ’, ’YASEIE’, ’VAWLKM’, ’NLGPVL’, ’RTPKIQ’, ’EHSDLS’, \
’AEMEYL’, ’NYNTYR’, ’TAELIT’, ’HTEIIE’, ’AEKLFD’, ’LAEAIG’, \
’STVIME’, ’GERGFF’, ’VYSRHP’, ’YFQINN’, ’SWVIIE’, ’KGENFT’, \
’STVIWE’, ’STYIIE’, ’QTNLYG’, ’HYQWNQ’, ’IEKVEH’, ’KMFFIQ’, \
’ILENIS’, ’FFWRFM’, ’STVINE’, ’STVAIE’, ’FLKYFT’, ’FGELFE’, \
’STVILE’, ’WSFYLL’, ’LMSLFG’, ’FVNQHL’, ’STVIAE’, ’KTVIIE’, \
’MYWIIF’]
The biologist’s question is which method, based on decision trees or based on gram-
matical inference will achieve better accuracy.
As regards decision trees approach, classification and regression trees (CART),
a non-parametric decision tree learning technique, has been chosen. For this purpose
we took advantage of a scikit-learn’s3 optimized version of the CART algorithm:
3 Scikit-learnprovides a range of supervised and unsupervised learning algorithms via a consistent

interface in Python. It is licensed under a permissive simplified BSD license. Its web page is http://
scikit-learn.org/stable/.
14 1 Introduction
from sklearn import tree

from sklearn.externals.six import StringIO
from functools import partial
Sigma = set(list("NMLKIHWVTSRQYGFEDCAP"))
idx = dict(zip(list(Sigma), range(len(Sigma))))
def findACC(f):
score = 0
for w in Test_pos:
if f(w):
score += 1
for w in Test_neg:
if not f(w):
score += 1
if score == 0:
return 0.0
else:
return float(score)/float(len(Test_pos) + len(Test_neg))
def acceptsBy(clf, w):

return clf.predict([map(lambda c: idx[c], list(w))])[0] == 1
X = []
Y = []
for x in Train_pos:
X.append(map(lambda c: idx[c], list(x)))
Y.append(1)
for y in Train_neg:
X.append(map(lambda c: idx[c], list(y)))
Y.append(0)
clf = tree.DecisionTreeClassifier()
clf = clf.fit(X, Y)
print findACC(partial(acceptsBy, clf))
As for GI approach, it was the induction of a minimal NFA choice, since the
sample is not very large.
from functools import partial
from FAdo.common import DFAsymbolUnknown
def acceptsBy(aut, w):

try:
return aut.evalWordP(w)
except DFAsymbolUnknown:
return False
S_plus, S_minus = set(Train_pos), set(Train_neg)

k = 1
while True:
print k,
A = synthesize(S_plus, S_minus, k)
if A:
print findACC(partial(acceptsBy, A))
print A.dotFormat()
break
k += 1
The resulting automaton is depicted in Fig. 1.3. The ACC scores for the obtained deci-
sion tree and NFA are equal to, respectively, 0.616 and 0.677. It is worth emphasizing,
N, M, Y E, C, T
E, N, W, V, S, Q F, L
q1 F, I
F, N, L, Y N, H, W, V, S, Y
G, E, D, A, L, V, T, S, Q I
F, M, I, T, S, Y
N, W, T
F, A, K, W, V, R q3 q2
F, A, N, L, K, I, V, Q, Y
q0 G, D, W
G, D, N, M, L, H
G, E, C, A, T, Q
Fig. 1.3 An NFA consistent with the (Train_pos, Train_neg) sample
though, that McNemar’s test does not reject the hypothesis that the two algorithms
have the same error rate at significance level 0.05 (the computed statistic was equal
to 0.543, so the null hypothesis might be rejected if α ≥ 0.461).
1.4 Bibliographical Background
Different mathematical models of grammatical inference come from various papers.

Language identification in the limit has been defined and studied by Gold (1967). Two
other models also dominate the literature. In the query learning model by Angluin
(1988), the learning algorithm is based on two query instructions relevant to the
unknown grammar G: (a) membership—the input is a string w and the output is
‘yes’ if w is generated by G and ‘no’ otherwise, (b) equivalence—the input is a
grammar G and the output is ‘yes’ if G is equivalent to G and ‘no’ otherwise. If the
answer is ‘no’, a string w in the symmetric difference of the language L(G) and the
language L(G ) is returned. In the probably approximately correct learning model
by Valiant (1984), we assume that random samples are drawn independently from
examples and counterexamples. The goal is to minimize the probability of learning
an incorrect grammar.
The complexity results devoted to GI from combinatorial perspective were dis-
covered by Gold (1978), for DFAs, and Angluin (1976), for regular expressions. The
hardness of NFA minimization was studied by Meyer and Stockmeyer (1972) and
Jiang and Ravikumar (1993). The information on the number of k-state DFAs and
NFAs is taken from Domaratzki et al. (2002). Some negative results about context-
free languages, which also are mentioned in Sect. 1.1.4, come from papers: Charikar
et al. (2005), Hunt et al. (1976), and books: Hopcroft et al. (2001), Higuera (2010).
An algorithm for finding the minimal DFA consistent with a sample S = Σ ≤n was
constructed by Trakhtenbrot and Barzdin (1973).
Beside regexes, DFAs, NFAs, and CFGs, there are other string-rewriting tools that
can be applied to the grammatical inference domain. Good books for such alternatives
16 1 Introduction
are Book and Otto (1993) and Rozenberg and Salomaa (1997). Some positive GI
results in this context can be found in Eyraud et al. (2007).
Parsing with context-free grammars is an easy task and is analyzed in many books,
for example in Grune and Jacobs (2008). However, context-sensitive membership
problem is PSPACE-complete, which was shown by Kuroda (1964). In fact, the
problem remains hard even for deterministic context-sensitive grammars.
A semi-incremental method described at the end of Sect. 1.1 was introduced by
Dupont (1994). Imada and Nakamura (2009) also applied the similar approach of
the learning process with a SAT solver.
Statistical tests given in this chapter were compiled on Alpaydin (2010). Another
valuable book, especially when we need to compare two (or more) classifiers on
multiple domains, was written by Japkowicz and Shah (2011).
A list of practical applications of grammatical inference can be found in many
works; the reader can refer to Bunke and Sanfelieu (1990), de la Higuera (2005),
Higuera (2010), and Heinz et al. (2015) as good starting points on this topic. The
first exemplary application (peg solitaire) is stated as the 48th unsolved problem
in combinatorial games Nowakowsi (1996). This problem was solved by Moore
and Eppstein (2003) and we have verified that our automaton is equivalent to the
regular expression given by them. The second exemplary application is a hypothetical
problem, though, the data are real and come from Maurer-Stroh et al. (2010).
References
Alpaydin E (2010) Introduction to machine learning, 2nd edn. The MIT Press
Angluin D (1976) An application of the theory of computational complexity to the study of inductive
inference. PhD thesis, University of California
Angluin D (1988) Queries and concept learning. Mach Learn 2(4):319–342
Book RV, Otto F (1993) String-rewriting systems. Springer, Text and Monographs in Computer
Science
Bunke H, Sanfelieu A (eds) (1990) Grammatical inference. World Scientific, pp 237–290
Charikar M, Lehman E, Liu D, Panigrahy R, Prabhakaran M, Sahai A, Shelat A (2005) The smallest
grammar problem. IEEE Trans Inf Theory 51(7):2554–2576
de la Higuera C (2005) A bibliographical study of grammatical inference. Pattern Recogn
38(9):1332–1348
de la Higuera C (2010) Grammatical inference: learning automata and grammars. Cambridge Uni-
versity Press, New York, NY, USA
Domaratzki M, Kisman D, Shallit J (2002) On the number of distinct languages accepted by finite
automata with n states. J Autom Lang Comb 7:469–486
Dupont P (1994) Regular grammatical inference from positive and negative samples by genetic
search: the GIG method. In: Proceedings of 2nd international colloquium on grammatical infer-
ence, ICGI ’94, Lecture notes in artificial intelligence, vol 862. Springer, pp 236–245
Eyraud R, de la Higuera C, Janodet J (2007) Lars: a learning algorithm for rewriting systems. Mach
Learn 66(1):7–31
Gold EM (1967) Language identification in the limit. Inf Control 10:447–474
Gold EM (1978) Complexity of automaton identification from given data. Inf Control 37:302–320
Grune D, Jacobs CJ (2008) Parsing techniques: a practical guide, 2nd edn. Springer
References 17
Heinz J, de la Higuera C, van Zaanen M (2015) Grammatical inference for computational linguistics.
Synthesis lectures on human language technologies. Morgan & Claypool Publishers
Hopcroft JE, Motwani R, Ullman JD (2001) Introduction to automata theory, languages, and com-
putation, 2nd edn. Addison-Wesley
Hunt HB III, Rosenkrantz DJ, Szymanski TG (1976) On the equivalence, containment, and covering
problems for the regular and context-free languages. J Comput Syst Sci 12:222–268
Imada K, Nakamura K (2009) Learning context free grammars by using SAT solvers. In: Proceed-
ings of the 2009 international conference on machine learning and applications, IEEE computer
society, pp 267–272
Japkowicz N, Shah M (2011) Evaluating learning algorithms: a classification perspective. Cam-
bridge University Press
Jiang T, Ravikumar B (1993) Minimal NFA problems are hard. SIAM J Comput 22:1117–1141
Kuroda S (1964) Classes of languages and linear-bounded automata. Inf Control 7(2):207–223
Maurer-Stroh S, Debulpaep M, Kuemmerer N, Lopez de la Paz M, Martins IC, Reumers J, Morris
KL, Copland A, Serpell L, Serrano L et al (2010) Exploring the sequence determinants of amyloid
structure using position-specific scoring matrices. Nat Methods 7(3):237–242
Meyer AR, Stockmeyer LJ (1972) The equivalence problem for regular expressions with squaring
requires exponential space. In: Proceedings of the 13th annual symposium on switching and
automata theory, pp 125–129
Moore C, Eppstein D (2003) One-dimensional peg solitaire, and duotaire. In: More games of no
chance. Cambridge University Press, pp 341–350
Nowakowski RJ (ed) (1996) Games of no chance. Cambridge University Press
Rozenberg G, Salomaa A (eds) (1997) Handbook of formal languages, vol 3. Beyond words.
Springer
Trakhtenbrot B, Barzdin Y (1973) Finite automata: behavior and synthesis. North-Holland Publish-
ing Company
Valiant LG (1984) A theory of the learnable. Commun ACM 27:1134–1142
Chapter 2
State Merging Algorithms
2.1 Preliminaries
Before we start analyzing how the state merging algorithms work, some basic func-
tions on automata as well as functions on the sets of words have to be defined. We
assume that below given routines are available throughout the whole book. Please
refer to Appendixes A, B, and C in order to familiarize with the Python program-
ming language, its packages relevant to automata, grammars, and regexes, and some
combinatorial optimization tools. Please notice also that we follow the docstring con-
vention. A docstring is a string literal that occurs as the first statement in a function
(module, class, or method definition). Such string literals act as documentation.
from FAdo.fa import *
def alphabet(S):
"""Finds all letters in S
Input: a set of strings: S
Output: the alphabet of S"""
result = set()
for s in S:
for a in s:
result.add(a)
return result
def prefixes(S):
"""Finds all prefixes in S
Output: the set of all prefixes of S"""
result = set()
for s in S:
for i in xrange(len(s) + 1):
result.add(s[:i])
return result
def suffixes(S):
"""Finds all suffixes in S
Output: the set of all suffixes of S"""
© Springer International Publishing AG 2017 19

W. Wieczorek, Grammatical Inference, Studies in Computational Intelligence 673,
DOI 10.1007/978-3-319-46801-3_2
Another random document with
no related content on Scribd:
"Yes, ma'am, indeed it is very lonely sometimes," Salome acknowledged, "but I don't mind
that much. I have plenty to do, keeping the cottage clean and tidy, and preparing father's
meals, mending his clothes, and seeing to the flowers in the garden."
"How busy you must be. And you have lost your mother, poor child."
Salome pointed to a green mound at a little distance, whilst her brown eyes filled with
tears.
"She was such a good mother," she said softly, "oh, such a very good mother! And I was
such a fretful, tiresome child. I used to grieve her so often, and I can't bear to think of it
now."
She paused, but, encouraged by the sympathy on her companion's face, she continued,
"She used to be so patient with me when I was naughty and grumbled because I was not
able to run about and play like other children. And, until she lay dying, I never thought
how sorry I must have made her, and what a selfish girl I'd been. Then, I would have given
anything if I'd been different, but it was too late." And the repentant tears streamed down
Salome's cheeks.
"Don't grieve," said Mrs. Fowler, a little huskily, for she was much touched at the other's
evident remorse.
"I am sure Miss Margaret never treated you, ma'am, as I used to treat my mother!"
Salome exclaimed.
Mrs. Fowler was silent as she acknowledged to herself that Margaret had always been
patient and considerate when she had been an exacting invalid.
"I suppose your father is out in his fishing boat?" she asked by way of changing the
conversation.
"No, ma'am," Salome replied, the look of grief deepening on her face.
"Let us go into the church and hear Miss Conway play," Mrs. Fowler said, rising as she
spoke. "I hear Margaret's lesson is at an end. Ah, here comes the Vicar. How do you do,
Mr. Amyatt?"
"I am glad to see you are better, Mrs. Fowler," the Vicar exclaimed. "What, you here,
Salome? Don't go away; I want Mrs. Fowler to hear you sing."
Salome smiled, and blushed. She followed the others into the church and seated herself in
a pew near the door, whilst the Vicar pointed out beauties in the architecture of the
building to his companion, which she had failed to notice. Miss Conway was at the organ,
playing "The Heavens are telling," and when the last notes died away the Vicar beckoned
to Salome, who swung herself up the aisle on her crutches, and, at his request, consented
to sing.
"I will play the accompaniment," Miss Conway said, smiling encouragingly at the lame girl,
who felt a little shy at being called upon to sing alone. "What shall it be?" she inquired.
"Whatever you please, miss," Salome answered.
"Oh, no! You must choose," the Vicar declared decidedly.
"Then I will sing 'Abide with Me.'"

Mrs. Fowler and Margaret considerately withdrew to a side seat so that the sight of them
should not embarrass the singer, and Mr. Amyatt followed them. Salome stood a little
behind Miss Conway, who softly played the accompaniment of the hymn:
"Abide with me; fast falls the eventide;

The darkness deepens; Lord, with me abide;
When other helpers fail, and comforts flee,
Help of the helpless, O abide with me."
Salome's fresh, sweet voice rang clearly through the dim church, and its tender tones
touched the hearts of her audience. She was very fond of "Abide with Me," for it had been
her mother's favourite hymn, and to-night she sang her best.
"Heaven's morning breaks, and earth's vain shadows flee;

In life, in death, O Lord, abide with me."
The beautiful voice died lingeringly away, and for a few minutes there was a complete
silence. Then Mrs. Fowler rose, and coming eagerly forward, took Salome by the hand,
whilst she thanked her for giving her such a "rare treat" as she called it.
Margaret was delighted to see what a favourable impression her lame friend had evidently
made upon her mother, and great was her surprise when, on their all adjourning to the
churchyard, Mrs. Fowler asked Salome to come and see them at Greystone.
"I think you would be able to get as far as that, don't you?" she said with a winning smile.
"I should like you to come and sing to me. Will you, one evening?"
"Oh, yes," Salome replied. She had never been inside the doors of Greystone in her life,
though she had often desired to see what the house was like, having been told it was a
fine place.
"Then that is settled. I shall expect you."
Mrs. Fowler nodded and turned away, followed by Miss Conway, and Margaret who looked
back to wave her hand in farewell as she disappeared through the churchyard gate. The
Vicar accompanied them thus far, then turned back to speak a few words to Salome. The
village lad who had been employed to blow the organ had taken a short cut homewards
over the low wall.
"You sang remarkably well to-night," Mr. Amyatt said, "I felt quite proud of my pupil. You
showed excellent taste, too, in the hymn you chose. It was most suitable for the occasion.
I wonder if you know the circumstances under which that hymn came to be written?"
"No," Salome rejoined, shaking her head, "I don't know, sir."
"Then I will tell you. It was composed more than fifty years ago by a sick clergyman of the
name of Lyte, at a little fishing town called Brixham, in South Devon. He had become so
seriously ill that the doctors had ordered him abroad for his health's sake, and, after
service on the Sunday evening, prior to his leaving England, he went down to the sea-
shore, sad at heart, for he was convinced that he had spoken to his parishioners, who
were very dear to him, for the last time. He was sorrowful and low-spirited, but, by-and-
by, the remembrance that his Saviour was ever near to help and sustain him brought him
consolation. After watching the sunset, he went home, and immediately wrote the
beautiful hymn you sang to-night."
Salome had listened with deep interest, and she exclaimed earnestly: "Oh, Mr. Amyatt, I
am glad you have told me this. I shall love 'Abide with Me' better than ever now."
The Vicar smiled, then pointed towards the sea, over which a soft summer mist was
creeping.
"It is time for you to go home," he reminded her. "Where is your father this evening?"
"At the 'Crab and Cockle,' sir."
He shook his head sadly, but refrained from questioning her further. He saw she was
thinner than she had been a few months previously, and wondered if she was sufficiently
well fed, or if Josiah Petherick expended the money he should have spent on his home, on
the friends he met at the inn. As he watched the little girl swinging herself slowly down the
hill by the aid of her crutches, he felt very grieved and troubled on her account.
"What a curse this drink is!" he thought. "And it's a curse that creeps in everywhere, too."
In the village that afternoon, he had been told that Mr. Fowler had summarily dismissed a
groom who had been discovered with a bottle of beer in the stable, and he had listened to
various comments upon the strict notions of the master of Greystone. Most of the villagers
were inclined to think that the man's fault in disobeying his master's rule that no
intoxicating liquor should be brought on the premises might have been overlooked, as it
was his first offence, whilst some few argued that Mr. Fowler had acted rightly.
As Salome passed the "Crab and Cockle" on her way home, she heard sounds of hilarity
within, and recognised her father's voice singing a rollicking sea song. She sighed,
remembering how, during his wife's lifetime, Josiah had been a member of the church
choir; it appeared unseemly to her that a voice which once had been raised to the praise
and glory of God should lend itself to the entertainment of a set of half-drunken men in the
bar of a public-house. As she paused, involuntarily listening, a whiff of foul air, laden with
the mingled odour of smoke and beer, was wafted before her nostrils from the open
doorway, and she moved on with a sickening sense of shame and disgust, her heart heavy
as lead, her eyes smarting with tears. Oh, hers was a hard life, she thought bitterly.
Arrived at home, she laid a frugal supper of bread and cheese, and soon afterwards her
father reeled up the garden path and into the kitchen. Sitting down at the table, he helped
himself to bread and cheese in silence, and commenced eating, whilst his little daughter
took her accustomed place opposite to him.
"Where've you been?" he questioned. "I saw you pass the inn."
She told him how she had spent the evening, explaining that she had sung at the Vicar's
request, and that Mrs. Fowler had invited her to Greystone.
"I won't let you go there!" he cried. "I hate those new people! What did Mr. Fowler do
yesterday, but dismiss as honest a chap as ever lived, at a moment's notice, just because
he'd got a bottle o' beer in the stable! An' the man wasn't drunk either! No, you shan't go
nigh folks as treats their servants like that."
"Oh, father!" Salome exclaimed, disappointedly. She was wise enough, however, not to
pursue the subject. After a brief silence, she asked, with some timidity, "Father, have you
any money? Because, when Silas Moyle left the bread this afternoon, he said he couldn't
supply us with any more unless you paid him what you owe."
Silas Moyle was the one baker of the place, and the owner of the village shop, in which his
wife served. Josiah Petherick had formerly paid ready money for everything, but latterly he
had been spending at the "Crab and Cockle" what should have gone into Silas Moyle's
pocket. This was an additional trouble to Salome, but her father did not appear to care. He
was enraged, though, when he heard what the baker had said, and, as his creditor was not
present to bear the brunt of his indignation, Salome had to stand it instead. She turned
white when he swore at her, and sat perfectly still whilst he abused her roundly, but when
he called her extravagant she began to protest.
"Father, that's not fair of you! I'm as careful as ever I can be. We're obliged to have bread!
Won't you see Silas yourself? Perhaps he'll continue to supply us, if you can arrange to pay
him part of what we owe. Of course, he wants his money."
"He's another of your teetotal humbugs!" sneered the angry man.
"He isn't a humbug at all!" Salome retorted hotly, her indignation and sense of justice
overcoming her fear of her father; "but he did say he wasn't minded to wait for his money
when it was being squandered with that drunken crew at the 'Crab and Cockle.' Oh, father,
it was terrible for me to hear that, and I couldn't contradict him!"
With a fierce oath, Josiah pushed back his chair and rose from the table, declaring things
had come to a pretty pass when his own daughter, a mere child, thought fit to discuss him
with outsiders.
Salome broke into passionate weeping at this, whereupon he flung himself out of the
kitchen, and the next minute she heard his footsteps in the garden.
"He's gone to the 'Crab and Cockle' again," thought the unhappy little girl. "Oh, how could
he swear at me like that? Oh, how shall I bear it!" Presently she arose, put away the
supper things and then sat down by the open window to wait, as she knew she would have
to do, until the inn door was closed for the night, and her father would return. By-and-by,
the soft lap, lap of the sea had a soothing effect upon her troubled spirit, the peacefulness
of the summer night stole into her soul, and she murmured to herself the words of
consolation she had sung an hour or so before in the dim, old church:
"When other helpers fail, and comforts flee,

CHAPTER V.
Salome's Humiliation.
JOSIAH PETHERICK sat on the beach mending his fishing nets in the shade of a tall rock. It
was intensely hot, and there was scarcely a ripple on the glassy sea, whilst the sky was a
broad canopy of blue. Josiah was thinking deeply. That morning, consequent on the
information his daughter had given him on the previous evening, he had been to interview
Silas Moyle, and had induced the baker to allow him further credit. Never in his life before
had Josiah found himself in such a humiliating position, and he felt it all the more because
it was entirely his own fault. He had always prided himself on being able to pay his way,
and now he was not in the position to do so.
Glancing up from his work presently, the fisherman saw three figures come down to the
beach—a lady, a gentleman, and a small boy clad in a sailor's suit and broad-brimmed
straw hat. He knew them to be Mr. and Mrs. Fowler and their little son. He had often held
lengthy conversations with Gerald, who was always delighted to talk with anyone who
could tell him about the manifold wonders of the sea, but he had never spoken to either of
the boy's parents. Despite his disapproval of the strict teetotal principles of the master of
Greystone, he regarded that gentleman with considerable interest, and when Mr. Fowler
strolled up to him, and inquired from whom a boat might be hired, he answered him civilly,
"You can have a boat from me, if you like, sir; but there's no wind for sailing to-day."
"Perhaps you would row us around those high rocks yonder. My wife has a fancy to see
what lies beyond that point."
Josiah assented willingly, seeing an opportunity of earning a few shillings; and so it came
to pass that he spent a very pleasant and lucrative morning, returning home to dinner in
the best of spirits.
"The new folks at Greystone have a liking for boating," he informed Salome; "and see
here," tossing a half-crown as he spoke upon the table, "give that to Silas Moyle when he
calls with the bread this afternoon."
The lame girl's face brightened as she took up the coin, and looked at her father
questioningly.
"I saw Mr. and Mrs. Fowler and Master Gerald pass here on their way to the beach," she
said. "Did you take them out in your boat, father?"
"Yes. They treated me very fairly, I must admit that, an' Mrs. Fowler—she seems a nice
lady—spoke of you."
"Did she?"
"She said you had a lovely voice, an' that she was looking forward to hear you sing again.
I say, Salome, I shouldn't like to disappoint her, so if she really wants you to go and see
her, you may—" and Josiah, mindful of all he had said on the preceding night, avoided
meeting his little daughter's eyes as he made this concession.
"Oh, thank you, dear father," she cried. "I should like to go to Greystone so much."
"That little Master Gerald is a tiresome monkey," Josiah remarked. "He wouldn't sit still in
the boat at first, though his mother kept on with him. At last his father spoke, an' after
that, there was no need to tell him to be quiet again. Mr. Fowler looks a man as would
have his own way."
"Master Gerald is very disobedient, I know," Salome said, "and sometimes his governess
has great trouble with him. Miss Margaret says her mother spoils him."
"Then, 'tis a good job he's got a father who doesn't."

After dinner, Josiah went on with his interrupted work of mending his fishing nets, whilst
Salome tidied up the cottage and waited for Silas Moyle.
The baker looked gratified as he took the half-crown the lame girl tendered him, for he had
not expected to be paid even a small part of his account.
"That's right," he said, as he pocketed the money; "it appears I did some good by
speaking yesterday. Look here, my dear, you must try to keep that father of yours up to
the mark. Can't you make him stay at home of an evening?"
The little girl shook her head, and looked distressed as she replied, "I'm afraid not, Mr.
Moyle."
"He's not at the 'Crab and Cockle' now, I s'pose?"
"No, he's on the beach mending his nets; and this morning he took Mr. and Mrs. Fowler
and their little boy for a row in his boat."
"It's a pity Mr. Fowler can't get your father to his way of thinking—about drink, I mean. I
say the new folks at Greystone set an example that many in Yelton might follow with
advantage. Theirs is a teetotal household, I'm told."
"So I've heard," Salome responded.
Silas Moyle nodded kindly, and took himself off, whilst Salome locked up the cottage and
joined her father on the beach. She told him the baker had been pleased to receive the
half-crown, and then tactfully changed the subject. Josiah and his daughter were always
excellent friends when the former had not been drinking.
"Look!" Salome exclaimed suddenly, "There's Master Gerald. Why, he seems to be alone.
He sees us."
The child came running towards them, laughing as he stumbled over the rough shingles,
his face aglow with excitement, his broad-brimmed sailor's hat at the back of his head,
revealing the fair curls which clustered thickly around his brow.
"I've run away," he cried merrily. "I wanted Miss Conway to bring me down to the beach,
but she would not—the disagreeable thing! She said it was too hot, and I must stay in the
garden. So I came by myself."
"Doesn't Miss Conway know where you are?" Salome inquired.
"No one knows," he replied proudly. "I can take care of myself."
"I'm not so sure of that, young gentleman," Josiah remarked, with a chuckle of
amusement at Gerald's air of importance.
"It was naughty of you to run away," Salome told him in a tone of reproof.
The child made a grimace at her, and ran off towards some rocks which the receding tide
had left uncovered.
"He's a pretty handful," Josiah exclaimed, shaking his head.
"I expect someone will be here looking for him soon," said Salome. "I hope so, for his
mother will be anxious if she does not know where he is, and she is not strong."
But nobody came in search of Gerald, who at last disappeared from sight beyond the
rocks. In spite of her father's assurance that the boy could come to no harm, the little girl
grew uneasy about him; and, by-and-by, rose and went to make certain he was safe. She
found him lying flat on the wet beach, gazing into a pool between two rocks at some
beautiful anemones; and tried to induce him to retrace his footsteps, but all to no purpose.
In vain she told him that his mother would be worried about him, and that his father would
be angry. The wayward child would pay no attention to her.
"What's it to do with you?" he demanded rudely. "Mind your own business, if you please."
As he absolutely refused to return, Salome left him with the intention of persuading her
father to interfere; but, to her dismay, she found Josiah had deserted his nets, and as the
key of the cottage door was in her pocket, she knew he had not gone home. In all
probability he had betaken himself to the "Crab and Cockle" to obtain a drink. Whilst she
was hesitating how to act, much to her surprise, Gerald appeared around the rocks and
joined her. He was tired of the beach, he declared, and wanted to see her flowers, so she
allowed him to accompany her home. And thus it was that the young tyrant was
discovered in Salome's garden half-an-hour later by his much-tried governess.
Poor Miss Conway! She almost wept with joy on finding Gerald in safety, and insisted on
his return to Greystone immediately. She led him away in triumph, paying no attention to
his request that he might be allowed to remain a little longer.
Josiah did not return for his tea, so after waiting some time, Salome had hers, and then
seated herself under the porch with her knitting. There Margaret Fowler found her as the
evening was drawing in.
"Mother has sent me to thank you for taking such good care of Gerald this afternoon,"
Margaret said as she complied with the lame girl's invitation to sit down opposite to her.
"He is a very tiresome, disobedient boy, for father had told him never to go down on the
beach by himself. He is not to be trusted. Father has punished him for his naughtiness by
ordering him to bed. It was quite a shock to poor Miss Conway when she found Gerald was
nowhere on the premises."
"I noticed she looked pale," Salome said. "I am afraid Master Gerald is very troublesome."
"Troublesome! I should think he is. It was kind of you to look after him, Salome. I have a
message from my mother to know if you can come to see us to-morrow. Do try to come."
"Oh, I should like to!" Salome cried, her eyes sparkling with excitement.
"Then, will you manage to be at Greystone by five o'clock?"
"Yes, miss, if all's well. Oh, please thank Mrs. Fowler for asking me!"
"Mother wants to hear you sing again. She has taken quite a fancy to you, and I am so
glad."
"I think your mother is the prettiest, sweetest lady I ever saw," the lame girl said
earnestly. "How dearly you must love her, Miss Margaret."
"Yes," Margaret answered soberly, "but I do not think she cares for me much. Gerald is her
favourite, you know. Oh, I'm not jealous of him, but I can't help seeing that though he
teases and worries her, and I do all I can to please her, she loves him much better than
she has ever loved me."
Salome was surprised, and pained by the look of sadness on her companion's face.
"Perhaps your mother shows her affection more to Master Gerald because he's so much
younger than you," she suggested. "I cannot believe she loves him better really."
Margaret made no reply to this, but by-and-by she said, "We have had several fusses at
home these last few days. Did you hear that father dismissed one of the men-servants for
bringing beer into the stable?"
"Yes, I heard about it. I think Mr. Fowler was quite right," Salome declared decidedly.
"Do you? I'm glad to hear you say that. Father always means to do right, I am sure. He is
a teetotaler himself, you know, and so are we all, for that matter."
At this point in the conversation the garden gate clicked, and Josiah strode up the path
and hurried past the little girls into the cottage. His bronzed face was crimson; and he
walked somewhat unsteadily; but he was sufficiently sober to realise that his wisest plan
was to take no notice of his little daughter's visitor.
Pitying Salome from the depths of her heart, Margaret rose, saying it was time for her to
go home. The lame girl followed her silently to the garden gate, where they stood for a few
minutes talking.
"You'll be sure to come to-morrow, won't you?" Margaret said earnestly.
"Yes, miss," was the grave reply, "if I possibly can; I hope nothing will prevent it, but—you
see how it is with him sometimes," and she pointed towards the cottage.
"Yes," Margaret admitted. "Oh, I'm so sorry! He must be a terrible trial for you. May God
help you, Salome."
"He does help me," the lame girl replied, "I couldn't bear it alone. Oh, how I wish my
father was a teetotaler like yours."
"I wish so, too."
"I had hoped you would never find out about my poor father being a drinker, but I might
have known that sooner or later you would learn the truth. Oh, miss, don't, please don't
think, he's altogether a bad man. He isn't! When he's sober, there's not a kinder or better
man in the world. But when the drink gets hold of him, he isn't himself at all." And Salome
laid her head on the top rail of the gate and sobbed heartbrokenly.
"Oh, don't cry so!" Margaret said imploringly, her own eyes full of tears. "Oh, perhaps he'll
give up the drink some day."
"I don't know, miss, I'm afraid he won't. He gets worse instead of better. The Vicar has
spoken to him, but that's done no good. He has only come home for supper now;
afterwards he'll go back to the 'Crab and Cockle.' But there, I mustn't cry any more, or
he'll notice it!"
SALOME LAID HER HEAD ON THE TOP RAIL OF THE GATE
AND SOBBED HEART-BROKENLY.
"Good-bye, Salome! Mind you come to-morrow."
"Oh, yes! I hope I shall. Oh, miss, I feel so ashamed that you should have seen my father
to-night!"
"There's nothing for you to be ashamed about. I think you're the pluckiest girl I know.
Good night!" And Margaret ran off with a nod and a smile.
She slackened her speed soon, however; and as she went up the hill beyond the church
towards her home, paused now and again to look back the way she had come, and admire
the beautiful view. At the entrance to the grounds of Greystone she met her father, and
together they walked towards the house, whilst she told him of Josiah Petherick's condition
that evening.
"Oh, father, you are right to be a teetotaler!" she cried. "Drink is an awful thing!"
"It is indeed, my dear," he replied with a deep sigh. "I found Petherick a well-informed,
civil-spoken man, in fact I was favourably impressed with him this morning, and he talked
of his little daughter as though he really loved her. Drink can slay affection, though," he
concluded sorrowfully.
"It's dreadful it should, father!"
"When drink once gets hold of people, it makes them slaves, and kills their finest feelings.
I am very sorry for that poor Salome!"
"So am I. She is so brave, too, and sticks up for her father all she can. Oh, I think he
ought to give up the drink for her sake. I wonder—I wonder if it would be any good for you
to speak to him!" And Margaret looked wistfully and pleadingly into her father's face.
"I will consider the matter," he rejoined thoughtfully.
"Oh, father!" she cried, picturing afresh Salome's grief and humiliation, "What should I do,
if I had such a trouble as that poor lame girl has to bear?"
Mr. Fowler started, and a look of intense pain and trouble momentarily crossed his
countenance, but he answered quietly, "In that case, I hope you would ask God to support
and comfort you."
"As Salome does. I could not be patient like she is, though."
"I trust you would, my dear child."
"Well, I am not likely to be tried," and Margaret regarded her father with a look of
affectionate pride. She wondered at the sadness of the smile with which he returned her
glance; and his answer, gravely spoken, puzzled her not a little.
"We cannot tell how much our patience and our love may be tried," he said, "nor what
trials the future may hold for us. We can only pray that God will help and strengthen us in
our time of need."
CHAPTER VI.
Perfectly Happy.
"OH, I do hope she will come! It's nearly five o'clock, and she's not in sight yet. I wish I
had thought of watching from my bedroom window, I could have seen then when she left
the cottage."
The speaker, Margaret Fowler, started up from her seat beneath the lilac tree, and ran
across the lawn in the direction of the gate which led from the grounds of Greystone into
the road. Beneath the lilac tree sat Mrs. Fowler in a comfortably padded wicker chair, with
a small table laden with papers and magazines at her side. She glanced after her little
daughter with a slightly amused smile, then remonstrated with Gerald, who was playing
near by, for making a noise.
"You will give me a headache, if you keep on doing that," she said, as he cannoned two
croquet balls against each other. "Pray, be quiet!"
Gerald chose not to obey. He continued his game, utterly regardless of his mother's
command.
"Do stop, Gerald!" she exclaimed. "I really cannot bear that noise any longer. Oh, where is
Miss Conway? Why isn't she here to look after you? Gerald, to oblige me, find some other
amusement, there's a dear boy!"
"Why do you not obey your mother, sir?" demanded a stern voice. And suddenly the little
boy dropped the croquet-mallet from his hand, and turned to face his father.
"That's right, Gerald!" Mrs. Fowler said hastily. "He hasn't been doing anything wrong,
Henry," she continued, glancing apprehensively at her husband, "only—you know how
absurdly nervous I am—I can't bear any sharp, sudden noise. It's foolish of me, I know."
Gerald now ran after his sister, and Mr. Fowler stood with his hand on the back of his wife's
chair, looking, down at her with grave attention.
"You should make the boy obey you, my dear," he said. "Has not your visitor arrived yet?"
"No. Margaret has gone to the gate to see if she is coming. I thought we would have tea
out here, for it is cooler and pleasanter in the garden than in the house, and it will be more
informal. I should like you to hear this lame girl sing, Henry! I think I never heard a voice
which touched me so deeply as hers. But you are not listening—"
"I beg your pardon, my dear. I confess my thoughts were wandering. The fact is, to-
morrow I shall have to go up to town for a few days, and I would far rather remain at
home. But I am obliged to go."
"You can leave with an easy mind," his wife told him reassuringly. "I am really quite strong
now, and capable of managing the household, I believe I shall be better for something to
do. By the way, you cannot think how much I enjoyed my drive this morning to N—"
mentioning the nearest town. "I wanted some trifles from a draper's, and the shops were
much better than I expected. Oh! Here come the children. They are bringing Salome with
them."
Mrs. Fowler rose and greeted the lame girl very cordially, placing her in a chair next to her
own. Salome was looking her best, neatly attired in a clean cotton frock. There was a flush
born of excitement on her cheeks, and her brown eyes shone with a happy light as she
gave herself up to the enjoyment of the present hour.
Tea was served beneath the lilac tree, such a luxuriant tea as Salome had never partaken
of before, and everyone appeared determined that she should make a good meal—Gerald
pointing out to her the most delectable of the dainties which he pressed her to eat, for in
the depths of his selfish little heart, there was a warm spot for the lame girl who had so
often given him flowers from her garden when he had certainly not deserved them.
Salome was inclined to be a trifle shy at first of Mr. Fowler. From what she had heard of
him she had imagined he must be an exceedingly stern, strict sort of man, but he talked to
her so kindly and pleasantly that she soon grew at ease with him, and answered all the
questions he put to her unreservedly. She told him she had only been a member of the
choir during the last six months, and explained that she had not known she possessed a
really good voice until the Vicar had informed her that such was the fact.
"I always loved singing, even when I was a tiny thing," she said, "but I never thought of
joining the choir till one day when Mr. Amyatt suggested it. He was passing our cottage,
and heard me singing, and he came right in and said he would like me to come up to the
Vicarage and let him try my voice. Father said I might go, so I did, and the next Sunday, I
sang with the choir in church for the first time."
"You must not sing too much," Mr. Fowler remarked, "for you are very young, and might
permanently injure your voice if you strained it now. You must nurse it a bit."
"That's what Mr. Amyatt says," Salome replied with a smile, "and I'm very careful."
"It is a great gift to have a beautiful voice." Mr. Fowler looked with kindly interest at his
little guest as he spoke; then his eyes wandered to the crutches which she had placed on
the ground beside her chair, and she caught the swift glance of sympathy which crossed
his face, and from that moment, he stood high in her estimation.
"God is very merciful," he added softly, as though speaking to himself; "we are too apt to
forget that He never sends a cross without its compensation."
Salome was perfectly happy sitting there under the lilac tree, though she felt all the while
as though she must be in a wonderful dream. Mrs. Fowler, in her light summer dress, with
her fair hair and her lovely blue eyes, looked like a queen, she thought. Salome was more
and more impressed with her grace and charm on every fresh occasion on which she saw
her. How proud Miss Margaret must be of her mother! And how happy Miss Margaret must
be in such a beautiful home, with kind parents, and everything that heart could desire!
And yet, what was the meaning of that wistful look on her face; and why was Mr. Fowler's
countenance so grave, and almost stern in expression at times? Salome's eyes were
remarkably shrewd. She noticed how attentive Mr. Fowler was to his wife, almost seeming
to anticipate her wishes and read her thoughts; and she was surprised when he was called
away for a few minutes to see that Mrs. Fowler talked with greater freedom in his absence,
as though his presence put a restraint upon her.
As soon as all had finished tea, Margaret took Salome around the gardens, and afterwards
led the way into the house. She showed Salome her own room, the walls of which were
crowded with pictures and knickknacks. The lame girl had never seen such a pretty
bedroom before as this one, with its little white-curtained bed, and white-enamelled
furniture. Then Margaret opened a velvet-lined jewel case, and took out a small, gold
brooch in the shape of a shell, which she insisted upon fastening into the neck of her
visitor's gown.
"It is for you," she said, "I bought it with my own money, so you need not mind taking it. I
told mother I was going to give it to you. I want you to wear it for my sake, Salome."
"Oh, Miss Margaret, how kind of you! Thank you so much. But ought I to take it? Are you
sure Mrs. Fowler—"
"Oh, yes!" Margaret interposed eagerly. "Mother would like you to have it. She said she
thought it would be a very suitable gift for you. It is pretty, isn't it?"
"It is lovely!" was the enthusiastic reply. "I shall value it always, Miss Margaret, for your
sake," and there were tears of pleasure and gratitude in Salome's brown eyes as she
spoke.
"I am so very glad you like it," Margaret said earnestly; "but now, come downstairs to the
drawing-room."
Greystone appeared quite a palatial residence to the simple village girl, accustomed to her
cottage home. She noticed how soft and thick were the carpets, how handsome was the
furniture; and how everything in connection with the house had been done with a view to
comfort. A sense of awe crept over her, as she cast one swift glance around the spacious
drawing-room. Miss Conway was at the piano, but she ceased playing as the little girls
entered; and Mrs. Fowler, who was standing by the open window conversing with her
husband, turned towards them immediately and requested Salome to sing.
So Salome stood, leaning upon her crutches, in the centre of the room, and lilted, without
accompaniment, a simple little song she had often heard from her dead mother's lips. It
was a lullaby, and she sang it so sweetly and unaffectedly that her listeners were
delighted, and Mr. Fowler was surprised at the beauty of the voice which had had so little
training. She gave them several other quaint west-country ballads; and then, at Mrs.
Fowler's request, sang, "Abide with Me."
"I like that best," Margaret said, as she drew Salome down on a sofa by her side. "Why,
how you're trembling! And your hands are quite cold!"
"Poor child! We have made her nervous, I fear," Mr. Fowler remarked. "Used your mother
to sing, my dear?"
"Yes, sir, sometimes, and father used to sing in the choir, but he doesn't now. If you
please," she proceeded, glancing from one to the other hesitatingly, "I think I ought to go
home. Father promised to meet me outside the gate at seven o'clock, and it must be that
now."
"It is a little after seven," Mr. Fowler replied, glancing at his watch.
"Then I think I must go, sir."
"You must come again soon," Mrs. Fowler said eagerly. "Thank you so much, my dear, for
singing to us. You have given us very great pleasure."
"I am very glad," Salome rejoined simply and earnestly, "and I should like to tell you how
much I have enjoyed myself; and thank you for all your kindness to me."
True to his promise, Josiah Petherick was waiting for his little daughter in the road outside
the entrance to Greystone. He was perfectly sober, and as Salome caught sight of his
stalwart figure, her face lit up with pleasure.
"Well, have you had an enjoyable time?" he inquired, smilingly.
"Oh, yes," she answered, and proceeded to give him a detailed account of all she had
seen, and heard, and done. He admired Margaret's gift, and was secretly much gratified at
the attention and kindness his little girl had received from the new-comers. Much to her
relief, he accompanied her past the "Crab and Cockle," though it must be admitted, he cast
a longing glance in the direction of the open doorway through which the stale odour of
tobacco and beer was stealing forth as usual. And when they reached home, he followed
her into the cottage, and continued the conversation whilst she set about getting supper.
She feared he would take himself to the inn as soon as the meal was over, but, instead, he
sat down under the porch and gazed thoughtfully out to sea.
"That Mr. Fowler's a rare hand to talk," he remarked presently, when his little daughter
joined him. "That comes of being educated, I s'pose. He can argue a bit, he can."
"Can he?" Salome looked surprised. "How do you know, father?" she inquired.
"'Cause I was foolish enough to try to argue with him, my maid!"
"Oh! When was that?"
"This morning, on the beach."
"Oh!" she cried again, more and more astonished. "What did you argue about, father?"
She ventured to ask.
"Drink!" was the brief reply. And there was that in Josiah's manner which forbade further
questioning.
Salome nestled silently close to her father's side, her head resting against his arm, as she
thought how nice it was to have him there with her, quite himself, and how dearly she
loved him. She listened to the murmur of the sea, and tried to count the stars appearing in
the sky, whilst Josiah recalled the argument he had had with Mr. Fowler, in which, he was
obliged to admit, he had come off worst. At last, a deep sigh from Salome drew his
attention to her, and he asked what was amiss.
"Amiss?" she echoed in astonishment. "Nothing."
"But you sighed, my dear."
"Did I? Then it must have been for joy. I'm perfectly happy, perfectly! And so I should
always be, if there was no such place as the 'Crab and Cockle' in Yelton."
"Well, Salome, I've not been there to-night."
"No, you have not, dear father," she answered affectionately, "and that's why I'm so
perfectly happy. My mind's at rest!"
CHAPTER VII.
An Afternoon's Outing.
MR. FOWLER was obliged to breakfast at seven o'clock, which was an hour-and-a-half
before the usual breakfast hour at Greystone, on the morning following Salome's visit, as
it was his intention to catch the first train to London from N—, and in order to do that he
would have to leave home before eight o'clock, and drive several miles. His journey had
been discussed on the previous night, and he had said good-bye to Miss Conway and the
children then. But, when he entered the breakfast-room as the clock struck seven, he
found his little daughter awaiting him.
"Why, Margaret!" he exclaimed in pleased surprise as he kissed her. "I did not expect to
see you, my dear! You are not generally an early bird."
"I'm afraid I am rather sleepy-headed in the mornings, as a rule," she confessed, "but I
made up my mind last night that I would have my breakfast with you to-day, dear father,
and see you off. Now do try to eat as much as ever you can," she added practically, as the
servant appeared with a tray holding a couple of covered dishes and the coffee-pot.
Mr. Fowler laughed, as he seated himself at the table with Margaret opposite to him, and
said he would take her advice.
"I am sorry I have to go," he remarked, "but I have no choice in the matter, as my lawyer
wants to consult me upon important business. I shall leave your mother in your charge,
Margaret."
"In my charge?" Margaret said inquiringly, looking surprised. "But she is not ill now, father!
See how cheerful and bright she was last night. And she has taken several walks. Oh, she
is heaps better and stronger than she was! I don't think you need worry about her."
"Perhaps not; but, nevertheless, I want you to devote as much of your time as you can to
her during my absence. I have spoken to Miss Conway, and she has consented to give you
a holiday till I return. Had I not seen you this morning, Miss Conway would have explained
my wishes to you. I desire you to accompany your mother when she drives out, and when
she goes into the village, or down to the beach—in short, make yourself her companion,
my dear, until I return. Do you understand?"
"Yes, father, I think so," Margaret replied, impressed by his serious tone. "I expect mother
will be dull when you are gone, so I will do my best to brighten her up!"
"That's a good child!"
"Only, sometimes she much prefers to have Gerald with her to me!"
"I would rather she had you. Remember what I have said, Margaret. I hope I shall not be
away very long, but it will of course depend upon circumstances."
Mr. Fowler made an excellent breakfast, and afterwards went upstairs to say good-bye to
his wife, whilst Margaret waited for him in the hall. He kissed his little girl tenderly on his
return, then, it being quite time for him to leave, entered the carriage which was waiting at
the door, and was driven off. Margaret felt a little depressed as she listened to the sound of
the carriage wheels dying away in the distance, for she was exceedingly attached to her
father, and home did not seem like home without him.
Knowing her mother must be awake, she went upstairs, and knocked at her bedroom door.
On being told to come in, to her surprise, Mrs. Fowler declared her intention of getting up
to breakfast.
"But do you feel well enough?" Margaret asked, for up to the present Mrs. Fowler, having
been an invalid, had always breakfasted in her own room at Greystone.
"Oh, yes!" was the quick response. "I'm tired of being treated like a sick person! What a
beautiful, bright morning it is, and not so hot, is it? Your father will have a fine day for his
journey."
"He did not want to go at all!"
"No. But that was foolish of him!"
"I think he did not like the thought of leaving you, mother. He feared you might be ill
whilst he was away."
"Oh, I am not likely to be ill again," Mrs. Fowler declared sanguinely. "I mean to throw off
my invalid-ish ways now, and surprise your father on his return. Send Ross to me,
Margaret, to help me dress."
"Shall I help you, mother? Do let me. I am sure I can do your hair as well as Ross."
Mrs. Fowler hesitated, but finally decided in favour of Ross; so Margaret went in search of
her. Ross was a well-mannered, good-tempered young woman who waited upon Mrs.
Fowler, and did the mending and sewing of the household. She expressed surprise and
pleasure on hearing that her mistress intended getting up and joining the family breakfast-
table.
"It shows how much stronger she feels, Miss Margaret," she said. "I've often thought if she
would bestir herself more she would be better in health and spirits."
Gerald grumbled loudly when he discovered that he was to do lessons whilst his sister was
to have a holiday. Why should Margaret be allowed nice drives with their mother when he
was obliged to stay at home and work. It was most unfair, he declared; and it may be
imagined that poor Miss Conway had rather a trying experience with her younger pupil on
the first day of his father's absence, when, in the afternoon, Mrs. Fowler and Margaret
drove to N—, and left him at home.
The road to N— lay through some most beautiful scenery, and Margaret thoroughly
enjoyed the drive. Now they were on an open common where the few trees to be seen
were stunted and grown one-sided, a fact which puzzled the little girl until it was explained
to her that the keen breeze blowing across the Atlantic was accountable for it, then she
remarked that the bare side of the trees was the one which faced the sea; now they had
left the common and were going down bill into a sheltered, wooded coomb, and by-and-by
the road led upwards again till the town of N— was reached, situated almost at the top of
the hill.
At the entrance to the town, Mrs. Fowler and Margaret got out of the carriage, and walked
up the main street—Fore Street it was called—looking into the shop windows. They had
paused outside a small china shop in which was some pretty pottery, when a familiar voice
addressed them in accents of pleasure and surprise.
"Can I believe my eyes? Who would have thought of meeting you here!"
Turning instantly they confronted a handsome, middle-aged lady, dressed as a widow,

whose comely face was wreathed in smiles. She was called Mrs. Lute, and had been a near
neighbour of theirs in London.
"Oh, how glad I am!" Mrs. Fowler exclaimed. "It is good to see you again! Are you staying
in the neighbourhood?"
"Yes; I have taken a furnished house at N— for two months. I saw it advertised, came to
see it, and the result is that here I am! Why, how well you look! And you were such a
wreck when you left town! Margaret, too, is looking all the better for the change of air! I
suppose you are still at Yelton?"
"Oh, yes! Why haven't you been to see us?"
"I have only been here a week. But, come, walk home with me, and have a cup of tea."
"I should like to, but I have some errands to execute. Oh, Margaret!" And Mrs. Fowler
turned to her little daughter eagerly. "Surely you could do the errands! See, here is the list
of what I want on this paper! Is your home far from here?" she inquired of Mrs. Lute.
"No, you must have passed it—a thatched, whitewashed house, with a porch covered with
clematis and roses."
"Oh, yes, I noticed it!" Margaret cried. "Mother, why don't you and Mrs. Lute drive back in
the carriage, and I will join you as soon as I have done the shopping?"
Thus it was arranged. Margaret was quite excited at meeting an old acquaintance, for Mrs.
Lute had long been on the friendliest terms with her neighbours in town. She was one of
the kindest of women, and had been exceedingly sympathetic during Mrs. Fowler's serious
illness in the spring.
When Margaret had executed her list of errands, she made her way to the whitewashed
house, outside which the carriage was waiting; and on being shown into the drawing-room
which faced the road, found her mother and Mrs. Lute seated there conversing happily.
"How warm the poor child looks!" the latter exclaimed. "Sit down in this comfortable chair,
my dear, and let me give you some tea; or would you rather have a glass of wine, for you
look tired, and—"
"Oh, no, thank you!" Margaret interposed hastily.
"Just as you like, my dear; but I persuaded your mother to take a little wine; I thought it
would do her good after her long drive, and I think it has refreshed her. Here's your tea,
my dear! Help yourself to cream and sugar, and do try this cake."
"Thank you, Mrs. Lute."
Margaret was startled for the moment to hear her mother had been drinking wine,
remembering how her father had refused to allow her to take it. She thought Mrs. Fowler
should have declined it; but the matter soon passed from her mind as Mrs. Lute began to
question her about Yelton.
"Everyone tells me it is a charming little village," Mrs. Lute said, "but your mother is not
enthusiastic about it. I think she is beginning to feel the lack of society. I have been telling
her she should be satisfied to have regained her health. She is looking wonderfully well."
Margaret, glancing at her mother, agreed with Mrs. Lute. No one would have guessed at
that moment that Mrs. Fowler had been an invalid so lately, for there was a pink colour in
her cheeks, and her blue eyes were shining with a happy light. She was as glad as was
Margaret to meet their old friend.
"You must pay us a visit at Greystone as soon as my husband returns," she said
hospitably, "and then you will be able to form your own ideas of Yelton and its inhabitants.
Margaret has struck up a friendship with a lame girl, Salome Petherick by name, and I
believe Gerald has picked acquaintance with several fishermen."
"Salome's father is a fisherman," Margaret remarked; "and oh, Salome has the most
beautiful voice you can possibly imagine, hasn't she, mother?"
"She certainly has. When you come to visit us, Mrs. Lute, you shall hear this Cornish
singing-bird. Poor girl, she is a sad cripple, yet she makes herself very useful, attends to
her father's cottage, and even does gardening!"
"She uses a pair of crutches as a rule," Margaret explained, "but when she is gardening,
she somehow manages to hop about on one, so that she has a hand free to work with.
Poor Salome! Her father drinks, and that is a great trouble to her."
"I should think so, indeed!" Mrs. Lute commented. "She ought to try to persuade him to
take the pledge. Total abstinence from all intoxicants is the only thing for some people."
"Father says," Margaret was beginning, when Mrs. Fowler somewhat abruptly changed the
conversation by inquiring for a mutual friend in town. It struck the little girl that her
mother did not wish her to air her father's teetotal views, so during the homeward drive
she recurred to the subject.
"Mother, I was going to tell Mrs. Lute that we are all teetotalers now," she said. "Don't you
want her to know?"
Mrs. Fowler hesitated and frowned slightly, refraining from meeting her little daughter's
gravely inquiring gaze.
"I suppose she will have to know, if she comes to stay with us at Greystone," she
responded in tones of annoyance. "I had forgotten your father's fad when I invited her."
"Oh, mother, don't call it a fad!" Margaret cried distressfully.
"That's what it is, child! Mrs. Lute is accustomed to take wine, yet no one can say she is
not a strictly temperate woman. Your father, I do not doubt, would like her to be a total
abstainer. Such nonsense! He used not to be so fastidious!" And Mrs. Fowler looked quite
angry.
Margaret made no answer. She had perfect faith in her father's judgment, but she felt
herself incapable of arguing the matter from his point of view.
On reaching home they found a telegram from Mr. Fowler, acquainting them with his safe
arrival in London. As Mrs. Fowler read it, the displeasure left her face for a softer, gentler
expression.
"How thoughtful he always is!" she exclaimed.
She was in exceedingly good spirits all the evening, and retired to rest apparently perfectly
well; but about midnight, Margaret was awakened by a sound in the room, and starting up
in bed, found her mother standing by her side in her night-gown, with a lighted candle in
her hand.
"What is it, mother? Are you ill?" The little girl inquired in alarm.
"No, but I am nervous, and cannot sleep! I wish your father had not gone! Did I frighten
you? I hope not. I felt I must have company."
Margaret was greatly astonished, for the thought had continually crossed her mind during
the day that Mrs. Fowler was relieved at her husband's absence. She jumped out of bed
immediately, and led her mother back to her own room.
"I will stay with you to-night, dear mother," she said gently. "You won't feel nervous then."
So mother and daughter lay down side by side, but not to sleep as yet, for the latter was
restless and sighed continually.
"You are sure you are not ill?" Margaret asked with loving anxiety.
"No, I am not ill, but I am very unhappy," was the response in a tone of great sadness.
"Oh, child, I wish you had a better mother!"
"You are the dearest mother in the world," Margaret cried earnestly.
"But very far from being the best. I am very troubled—no, I cannot tell you what about.
No, you couldn't help me. No one can."
"Yes, God can, mother," Margaret reminded her; then she quoted softly—
"When other helpers fail, and comforts flee,

Mrs. Fowler caught her breath with a little sob; but doubtless, the words of Salome's
favourite hymn comforted her, for presently, Margaret knew by her regular breathing that
she had fallen asleep.
The little girl lay awake wondering what trouble her mother could possibly have, or if she
was only nervous and imaginative; and it was not until the first streaks of dawn peeped
into the room that she slept too.
CHAPTER VIII.
An Awful Thing.
WHEN Margaret awoke, she was alone. At first she was surprised to find herself in bed in
her mother's room, but in a few moments, she remembered how that happened to be the
case. Before, however, she had time to dwell much upon the matter, the door opened and
her mother entered, fully dressed, bearing a breakfast tray in her hands, which she placed
on the dressing-table.
"Have I overslept myself?" Margaret inquired. "I am so sorry."
"You need not be, my dear," Mrs. Fowler replied, smiling as she came to the bedside and
kissed her little daughter. "You had a disturbed night on my account. How foolish it was of
me to be too nervous to sleep alone! I blame myself for spoiling your rest. But, see, I have
brought your breakfast, so sit up and eat it at once; after you have had it, you can dress
and come down on the beach with me."
Mrs. Fowler looked alert and well. She talked brightly whilst Margaret was taking her
breakfast, and pulling a letter out of her pocket, which she had received from her husband
by the morning's post, read it aloud. It merely told of his journey to town, and concluded
with his love to the children, and a hope that Mrs. Fowler would take care of herself.
"I shall not tell him how silly I was last night," she said. "I suppose I cannot be quite so
strong as I thought. My late illness played sad havoc with my nerves. It is such a glorious
day, Margaret, that I am sure we ought to spend it out of doors."
Margaret assented willingly, and went to her own room to dress. By-and-by, she and her
mother strolled down to the beach, and passed a pleasant morning in the welcome shade
of a big rock. And in the afternoon, Mrs. Fowler declared her intention of again driving to N
—.
"Won't you be very tired, mother?" Margaret asked dubiously. "You mustn't overdo it, you
know."
"Oh, I will be careful, my dear!" Mrs. Fowler rejoined. "But I want to get some things I
forgot yesterday. Meeting Mrs. Lute so unexpectedly put everything else quite out of my
head. Miss Conway and Gerald can accompany us."
It was not such a pleasant drive as the one of the previous day, for Gerald was tiresome,
and continually stood up in the carriage to look at different objects of interest which
attracted his attention. Miss Conway begged him to sit still, but he would not obey her.
And, at last, he was jolted into his mother's lap, much to her annoyance and to his
amusement. She declared she wished she had left him at home, and that it would be a
long time before she would take him for a drive again. Whereupon, he only laughed, for he
did not believe she meant what she said.
"Are you going to see Mrs. Lute, mother?" Margaret inquired as they neared the town.
"No, not to-day. I will get out at the bottom of Fore Street, and you others shall drive on a
little farther and return for me. No, I will not have you, Gerald! You are to stay with Miss
Conway and your sister."
Mrs. Fowler spoke with decision in her tones; she was evidently determined to do her
shopping alone.
Accordingly, she got out of the carriage at the entrance to the town, and the others saw
her go into a grocer's shop as they were driven on. When the carriage returned a quarter
of an hour later, she was standing waiting outside the same shop. The shopman came out
and placed a parcel in the carriage, then Mrs. Fowler took her seat and gave the order
—"Home." She seemed lost in deep thought during the remainder of the drive, and spoke
but seldom, paying slight attention to the conversation the others carried on. She was
evidently glad to reach Greystone.
"I expect she is really very tired," Margaret reflected, "but does not like to confess it." And
she was confirmed in this opinion when she saw how quiet and languid Mrs. Fowler
appeared during the evening. She did not request Miss Conway to play to her as she
usually did, but lay on the sofa with a book in her lap, yawning occasionally as though
weary of the day, so that neither Margaret nor the governess were surprised when she
declared her intention of going to bed early. She would not hear of Margaret sleeping with
her, however, but kissed both of her children good night in the drawing-room, and told
them not to disturb her when they went upstairs to bed.
It was only eight o'clock when Mrs. Fowler retired for the night. At half-past eight Gerald
was put to bed, after which Margaret and her governess sat down together to their supper.
Each seemed rather depressed, Miss Conway even more so than her little pupil.
"It is so dull without father," Margaret sighed. "I hope he will not stay away very long. Oh,
dear! I think mother is very, very tired to-night, don't you? I am afraid she has been doing
too much."
"I hope not," was the serious reply. "You did not walk far this morning, did you?"
"Oh, no! We were sitting down on the beach most of the time. Mother read the newspaper
and talked and seemed all right then."
"Did you see anything of Salome Petherick?"
"Nothing, though we stood outside her garden several minutes looking at her flowers. I
suppose she was busy in the cottage. Oh, Miss Conway, how I do wish Salome's father was
a teetotaler! I was telling Mrs. Lute about him yesterday, and she said Salome ought to try
to persuade him to take the pledge."
"I did not know that Mrs. Lute was a teetotaler," Miss Conway exclaimed, looking rather
surprised.
"She is not. Indeed, she offered me a glass of wine."

Textbook Grammatical Inference Algorithms Routines and Applications 1St Edition Wojciech Wieczorek Auth Ebook All Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Textbook Grammatical Inference Algorithms Routines and Applications 1St Edition Wojciech Wieczorek Auth Ebook All Chapter PDF

Uploaded by

Copyright:

Available Formats

Grammatical Inference: Algorithms,

Routines and Applications 1st Edition

Biota Grow 2C gather 2C cook Loucas

Computer Age Statistical Inference Algorithms Evidence

Computer Age Statistical Inference Algorithms Evidence

Inference for Heavy Tailed Data Applications in

Grouping Genetic Algorithms: Advances and Applications

Swarm Intelligence Algorithms: Modifications and

Data Mining Algorithms in C++: Data Patterns and

Wireless Algorithms Systems and Applications Sriram

The series “Studies in Computational Intelligence” (SCI) publishes new develop-

More information about this series at http://www.springer.com/series/7092

ISSN 1860-949X ISSN 1860-9503 (electronic)

© Springer International Publishing AG 2017

Printed on acid-free paper

This Springer imprint is published by Springer Nature

Sosnowiec, Poland Wojciech Wieczorek

3.3.2 Basic Notions of the Genetic Algorithm for GI. . . . . . . . . . 37

7.3 Our Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

CFG Context-free grammar

1.1 The Problem and Its Various Formulations

1.1.1 Mathematical Versus Computer Science Perspectives

© Springer International Publishing AG 2017 1

Fig. 1.1 A DFA accepting a b c

on both examples and counter-examples. Sometimes we will demand the smallest

1.1.2 Different Kinds of Output

Now, the language accepted by this grammar is {am bm cm : m ≥ 1}. It is worth

1.1.3 Representing Languages

Definition 1.2 Given two words u = u 1 u 2 . . . u n and v = v1 v2 . . . vm , u < v accord-

3. If r A and r B are regexes representing languages A and B, respectively, then

Definition 1.4 A non-deterministic finite automaton (NFA) is defined by a quintuple

The transition diagram of an NFA (as well as DFA) is an alternative way to

Definition 1.6 A context-free grammar (CFG) is defined by a quadruple G =

x for symbol A in a sentential form if A → x is a rule in P. We call this rewriting

1.1.4 Complexity Issues

The consecutive theorems are very important in view of computational intractability

Interestingly, the problem remains NP-complete even if r is required to be star-free

Let Σ be an alphabet, and k, n be positive integers. We will denote by I (input)

computational complexity primarily depends on the size of S+ .

1.2 Assessing Algorithms’ Performance

1.2.1 Measuring Classifier Performance

Table 1.1 Confusion matrix for two classes

• accuracy, ACC = (tp + tn)/N ,

1.2.2 McNemar’s Test

1.2.3 5 × 2 Cross-Validated Paired t Test

1.3 Exemplary Applications

As a concept, grammatical inference is a broad field that covers a wide range of

• Structural pattern recognition is a field in which grammatical inference has been

1.3.1 Peg Solitaire

The following problem has been formulated in one of CGT books:

Fig. 1.2 A DFA representing H

The resulting automaton is depicted in Fig. 1.2.

1.3.2 Classification of Proteins

One of the problems studied in bioinformatics is classification of amyloidogenic

’STVIYE’, ’VILLIS’, ’SQFYIT’, ’SVVIIE’, ’STVIII’, ’HLVYIM’, \

3 Scikit-learnprovides a range of supervised and unsupervised learning algorithms via a consistent

from sklearn import tree

def acceptsBy(clf, w):

def acceptsBy(aut, w):

S_plus, S_minus = set(Train_pos), set(Train_neg)

Fig. 1.3 An NFA consistent with the (Train_pos, Train_neg) sample

1.4 Bibliographical Background

Different mathematical models of grammatical inference come from various papers.