The Oxford Handbook of Experimental

Syntax Jon Sprouse

Visit to download the full and correct content document:
https://ebookmass.com/product/the-oxford-handbook-of-experimental-syntax-jon-spro
use/
 t h e ox f o rd ha n db o o k o f

EXPERIMENTAL
SYNTAX
 OX F OR D HA N DB O OK S I N L I NG U I ST IC S

RECENTLY PUBLISHED

THE OXFORD HANDBOOK OF REFERENCE

Edited by Jeanette Gundel and Barbara Abbott
THE OXFORD HANDBOOK OF EXPERIMENTAL SEMANTICS
AND PRAGMATICS
Edited by Chris Cummins and Napoleon Katsos
THE OXFORD HANDBOOK OF EVENT STRUCTURE
Edited by Robert Truswell
THE OXFORD HANDBOOK OF LANGUAGE ATTRITION
Edited by Monika S. Schmid and Barbara Köpke
THE OXFORD HANDBOOK OF NEUROLINGUISTICS
Edited by Greig I. de Zubicaray and Niels O. Schiller
THE OXFORD HANDBOOK OF ENGLISH GRAMMAR
Edited by Bas Aarts, Jill Bowie, and Gergana Popova
THE OXFORD HANDBOOK OF AFRICAN LANGUAGES
Edited by Rainer Vossen and Gerrit J. Dimmendaal
THE OXFORD HANDBOOK OF NEGATION
Edited by Viviane Déprez and M. Teresa Espinal
THE OXFORD HANDBOOK OF LANGUAGE CONTACT
Edited by Anthony P. Grant
THE OXFORD HANDBOOK OF LANGUAGE AND RACE
Edited by H. Samy Alim, Angela Reyes, and Paul V. Kroskrity
THE OXFORD HANDBOOK OF LANGUAGE PROSODY
Edited by Carlos Gussenhoven and Aoju Chen
THE OXFORD HANDBOOK OF LANGUAGES OF THE
CAUCASUS
Edited by Maria Polinsky
THE OXFORD HANDBOOK OF GRAMMATICAL NUMBER
Edited by Patricia Cabredo Hofherr and Jenny Doetjes
THE OXFORD HANDBOOK OF COMPUTATIONAL
LINGUISTICS
Second edition
Edited by Ruslan Mitkov
THE OXFORD HANDBOOK OF THE MENTAL LEXICON
Edited by Anna Papafragou, John C. Trueswell, and Lila R. Gleitman
THE OXFORD HANDBOOK OF ETHIOPIAN LANGUAGES
Edited by Ronny Meyer, Bedilu Wakjira, and Zelealem Leyew
THE OXFORD HANDBOOK OF EXPERIMENTAL SYNTAX
Edited by Jon Sprouse

For a complete list of Oxford Handbooks in Linguistics please see pp 675–78.

the oxford handbo ok of
...........................................................................................................

EXPERIMENTAL
SYNTAX
...........................................................................................................

Edited by
JON SPROUSE
 Great Clarendon Street, Oxford OX2 6DP,
United Kingdom
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide. Oxford is a registered trade mark of
Oxford University Press in the UK and in certain other countries
© editorial matter and organization Jon Sprouse 2023
© the chapters their several authors 2023
The moral rights of the authors have been asserted
First Edition published in 2023
All rights reserved. No part of this publication may be reproduced, stored in
a retrieval system, or transmitted, in any form or by any means, without the
prior permission in writing of Oxford University Press, or as expressly permitted
by law, by licence or under terms agreed with the appropriate reprographics
rights organization. Enquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxford University Press, at the
address above
You must not circulate this work in any other form
and you must impose this same condition on any acquirer
Published in the United States of America by Oxford University Press
198 Madison Avenue, New York, NY 10016, United States of America
British Library Cataloguing in Publication Data
Data available
Library of Congress Control Number: 2022935323
ISBN 978–0–19–879772–2
DOI: 10.1093/oxfordhb/9780198797722.001.0001
Printed and bound by
CPI Group (UK) Ltd, Croydon, CR0 4YY
Links to third party websites are provided by Oxford in good faith and
for information only. Oxford disclaims any responsibility for the materials
contained in any third party website referenced in this work.
 Contents
..............................

Preface ix
List of figures and tables xiii
The contributors xvii

PA RT I J U D G M E N T M ET HOD S I N
SY N TAC T IC T H E ORY
1. Acceptability judgments 3
Jon Sprouse

2. Acceptability judgments of binding and coreference:

Methodological considerations 29
Elsi Kaiser and Jeffrey Runner

3. (Quantifier) scope judgments 53

Kriszta Eszter Szendrői

4. Experimental syntax and linguistic fieldwork 97

Maria Polinsky

Annotated bibliography for Part I 127

PA RT I I AC QU I SI T ION M ET HOD S I N
SY N TAC T IC T H E ORY
5. Behavioral acquisition methods with infants 137
Laurel Perkins and Jeffrey Lidz

6. Behavioral acquisition methods with preschool-age children 171

Kristen Syrett
vi contents

7. Modeling syntactic acquisition 209

Lisa S. Pearl

8. Artificial language learning 271

Jennifer Culbertson

Annotated bibliography for Part II 301

PA RT I I I P SYC HOL I NG U I ST IC M ET HOD S

I N SY N TAC T IC T H E ORY
9. Self-paced reading 313
Masaya Yoshida

10. Eye-tracking and experimental syntax 333

Dave Kush and Brian Dillon

11. Speed–accuracy trade-off modeling and its interface with

experimental syntax 363
Stephani Foraker, Ian Cunnings, and Andrea E. Martin

12. Formal methods in experimental syntax 393

Tim Hunter

13. Investigating syntactic structure and processing in the

auditory modality 453
Mara Breen and Katy Carlson

14. Language-processing experiments in the field 491

Matthew Wagers and Sandra Chung

Annotated bibliography for Part III 513

PA RT I V N E U ROL I NG U I ST IC M ET HOD S I N
SY N TAC T IC T H E ORY
15. Electrophysiological methods 533
Jon Sprouse and Diogo Almeida

16. Hemodynamic methods 559

Jonathan R. Brennan
 contents vii

17. Aphasia and syntax 593

William Matchin and Corianne Rogalsky

Annotated bibliography for Part IV 635

18. The future of experimental syntax 643
The contributors

Index 665
 Preface
........................

The field of syntax has always been interdisciplinary. Part of this is simply the nature of
cognitive science—the immensity of the problem posed by human cognition requires
a concerted effort from multiple disciplines. And part of this is the nature of syntactic
theory: It mediates between sound and meaning, it is a theory of the representations
constructed during sentence-processing, and it is a theory of the end state for language
acquisition. As technology has advanced, so too have the methods that syntacticians
have brought to bear on the central questions of the field. The past two decades in par-
ticular have seen an explosion in the use of various experimental methods for probing
the syntax of human languages. This Handbook is an attempt to bring these strands
of research together into a single volume. I have three goals for this Handbook: (i) to
provide high-level reviews of the experimental work that has been driving the field of
experimental syntax, (ii) to inspire new research that will push the boundaries of the
theory of syntax, and (iii) to provide high-level methodological guidance for researchers
who wish to incorporate experimental methods into their own research. I hope readers
will agree that the contributors to this volume have created chapters that succeed in all
three goals.
For this handbook, I have intentionally defined experimental syntax in the broadest
possible terms—as the use of any (and all) experimental methods in service of syntactic
theory. I am aware that the term experimental syntax is sometimes used in a narrower
sense that is more or less synonymous with formal acceptability judgment experiments
(I have used it that way myself in my own work), but I believe this synonymy is merely
a symptom of the important role that acceptability judgments play in syntactic theory,
and not a meaningful delimiter of the types of methods that syntacticians can profitably
employ in their research. The space of possible methods is large—too large for any single
volume. In assembling this Handbook, I have chosen to focus on methods that are (i)
relatively well-understood, (ii) relatively practical in terms of the equipment required,
and (iii) relatively likely to yield information that is relevant to syntactic theory. All of
these choices are subjective. I do not intend the exclusion of any given method to mean
that it does not, or could not, fall under the broad definition of experimental syntax.
In fact, what I hope this Handbook shows is that this broader definition of experimen-
tal syntax is still in its infancy. We have not yet explored all of the methods that could
potentially contribute to theories of syntax, nor have we seen the full potential of the
methods that we have explored. As such, this handbook is aspirational—it is simulta-
neously a snapshot of the knowledge we have collected to date and a pointer to the kind
of work that will be possible in the future.
x preface

One critical component of all experimental work is a linking hypothesis—a hypothe-

sis that links the observed data to the unobserved theoretical constructs that are under
investigation. In experimental syntax, we need hypotheses that link each of the methods
discussed in this handbook back to syntactic theory. Because there is likely no method
that provides a direct link to syntactic theory (at least at our current level of technology),
for each and every method in this handbook, creating a linking hypothesis between
the data and syntactic theory entails creating (or investigating) a linking hypothesis
between syntactic theory and another component of the theory of language, such as
the theory of sentence-processing or the theory of language acquisition. Many of the
chapters in this Handbook discuss this issue in detail, so I will not belabor the point
here. The practical consequence of this is that there is a theoretical theme through-
out this handbook—the linking of syntactic theory to other components of a complete
theory of language. I have organized the Handbook around this theme. There are four
sections, each corresponding to the linking hypotheses necessary to leverage the meth-
ods in each section in service of syntactic theory: (i) judgment methods, which require
a link between the theory of offline judgments and syntax, (ii) acquisition methods,
which require a link between the theory of the language-acquisition process and syntax,
(iii) psycholinguistic methods, which require a link between the theory of sentence-
processing and syntax, and (iv) neurolinguistics methods, which require a link between
neurobiology, sentence-processing, and syntax.
A few notes on the organization of the Handbook are in order. First, I have asked Ox-
ford University Press to keep the references for each chapter with that chapter (and not
in a global reference list at the end of the Handbook). My hope is that this will allow the
chapters in this Handbook to truly serve as a guide for exploring the potential theoreti-
cal contributions that each method can make to syntax, with the reference lists serving
as a first reading list. Second, because these methods are not useful if they cannot be
learned by new researchers, I have asked the contributors of each chapter to create an
annotated list of resources for learning the method in their chapter. I have collated the
lists by section, compiling them into four stand-alone chapters that occur at the end of
each section. Finally, I have asked each contributor to write a mini-essay about what
they see as the future of experimental syntax. My hope is that these mini-essays will
provide inspiration to readers who are considering adopting experimental syntax into
their own research programs, and also serve as a sort of time capsule by which we can
measure the progress of the field in future years. I have collected these mini-essays into
a single chapter at the end of the handbook.
This volume could not exist without the talent, energy, and effort of innumerable
colleagues. First, I would like to thank the commissioning editor at Oxford University
Press, Julia Steer, for laying the foundation of this volume by encouraging me to explore
a broad definition of experimental syntax rather than a narrow review of work to date.
Second, I would like to thank each of the contributors for sharing both their visions
and their expertise. If this volume succeeds in any of its three goals, it will be because
 preface xi

of their hard work and dedication, both in writing their chapters and in doing the kind
of research that pushes the boundaries of the field. Finally, I would like to thank every-
one who has supported me throughout my career—advisors, collaborators, colleagues,
students, family, and friends. Science is a community effort. And I am grateful beyond
words for the community that I have somehow been given in this life.
 List of figures and tables
...............................................................................

Figures
1.1 The two predictions of the 2×2 design for whether-islands (left panel
and center panel), and the observed results of an actual experiment
(right panel) 17
1.2 Three demonstrations of the continuous nature of acceptability
judgments 21
2.1 Scene verification display from the experiment by Kaiser et al. (2009) 43
2.2 Picture selection display from the experiment by Kaiser et al. (2009) 44
3.1 Outcome of example test story from Conroy et al.’s (2009) TVJT task 87
7.1 Model of the acquisition process adapted from Lidz and Gagliardi (2015) 213
11.1 SAT function for one condition, illustrating the three phases of
processing 369
11.2 Idealized differences in the three phases of the SAT functions for two
conditions 370
11.3 Idealized differences in the finishing time distributions corresponding to
the SAT differences shown in Fig. 11.2 370
12.1 We can use surprisal to formulate a linking hypothesis which, taken
together with a probability distribution over sentences, produces
empirical predictions about sentence comprehension difficulty 401
12.2 Since surprisal can act as a test of probability distributions and
probability distributions can be seen as consequences of hypothesized
grammars, surprisal can act as a test of hypothesized grammars 402
12.3 Graphical illustration of lc-predict and lc-connect 431
13.1 Example item from Breen et al. (2010) designed to elicit naturalistic
productions 459
14.1 A sample itemset from Sussman and Sedivy (2003) 494
14.2 Some culturally specific illustrations created for the Chamorro
Psycholinguistics na Project 497
16.1 The BOLD signal 562
xiv list of figures and tables

16.2 Schematic representation of syntax-related brain regions of the left

hemisphere 566
16.3 Linking hypotheses connect properties of the grammar with neural
signals 582
17.1 Acceptability judgment data reproduced from Linebarger et al. (1983) 611
17.2 Functional neuroanatomy of language and working memory (WM) as
relevant to our proposal 618
17.3 A schematic of healthy and agrammatic sentence/phrase production
with respect to the dorsal and ventral pathways to articulation 622

Tables
3.1 Percentage of surface scope response for comprehension question 64
3.2 Summary of experimental findings of the language acquisition studies
reviewed in this paper 74
4.1 Experimental paradigm for studying subject preference,
morphologically ergative languages 108
7.1 The qualitative fit Yang discovered between the unambiguous data
advantage (Adv) perceived by a VarLearner in its acquisitional intake
and the observed age of acquisition (AoA) in children for six parameter
values across different languages 236
7.2 Optional infinitive examples in child-produced speech in different
languages, and their intended meaning 236
8.1 Summary of key artificial language learning methods 276
12.1 A first illustration of bottom-up parsing 417
12.2 The effect of center-embedding on bottom-up parsing 420
12.3 The effect of left-embedding on bottom-up parsing 421
12.4 The effect of right-embedding on bottom-up parsing 423
12.5 A first illustration of top-down parsing 424
12.6 The effect of center-embedding on top-down parsing 426
12.7 The effect of left-embedding on top-down parsing 427
12.8 The effect of right-embedding on top-down parsing 428
12.9 A first illustration of left-corner parsing 430
12.10 The effect of center-embedding on left-corner parsing 433
12.11 The effect of left-embedding on left-corner parsing 434
12.12 The effect of right-embedding on left-corner parsing 435
 list of figures and tables xv

16.1 Syntactic representations stand in a many-to-many relationship with

sentence-processing operations 564
16.2 Summary of brain regions related to syntax 565
16.3 Examples of phrases and sentences made of of real words or nonsense
pseudo-words 572
16.4 Hypothetical counts for utterances with intransitive verbs 580
17.1 Examples of stimuli from each condition 612
 The contributors
.......................................................

Diogo Almeida is an Associate Professor of Psychology at New York University Abu

Dhabi. His research capitalizes on behavioral and electrophysiological data (EEG and
MEG) to investigate questions about linguistic representations and processes at multi-
ple levels (phonology, morphology, and syntax). He holds an MA in cognitive science
from the École des Hautes Études en Sciences Sociales (2003) and a PhD in linguistics
from the University of Maryland (2009), and completed his post-doctoral training at
the University of California, Irvine.
Mara Breen is an Associate Professor in the Department of Psychology and Education
at Mount Holyoke College. Her research explores the role of prosody in speech per-
ception and production. Using behavioral techniques, eye-tracking, and event-related
potentials, she investigates how speakers use prosodic cues to provide meaning, how
listeners use prosody to comprehend, and how imagined prosody during reading can
affect understanding. In addition, she explores how sound cues are processed similarly
across music and language. Her work appears in journals such as Cognition, Journal
of Memory and Language, Journal of Experimental Psychology: General, and Language,
Cognition, and Neuroscience.
Jonathan R. Brennan is an Associate Professor of Linguistics and Psychology at the
University of Michigan, where he directs the Computational Neurolinguistics Labora-
tory. He received a PhD in linguistics from New York University in 2010 and completed
post-doctoral training at the Children’s Hospital of Philadelphia.
Katy Carlson is a Professor of English in the Department of English at Morehead State
University. Her research concentrates primarily on how prosody can affect sentence-
processing, with special interests in focus effects in ellipsis sentences and prosodic
influences on attachment. She studies both pitch accents and prosodic boundaries, and
has published in journals such as Language and Speech, Glossa, and Language, Cognition
and Neuroscience.
Sandra Chung is Distinguished Professor (emerita) of Linguistics at the University
of California, Santa Cruz. Her research investigates theoretical issues in syntax and
other areas through fieldwork on Chamorro and other Austronesian languages. She has
collaborated on research in semantics with William A. Ladusaw, and on research in psy-
cholinguistics with Matthew Wagers. Since 2009 she has been involved in a community-
based effort in the Northern Mariana Islands to upgrade the documentation of the
Chamorro language.
xviii the contributors

Jennifer Culbertson is a Professor in the Department of Linguistics and English Lan-

guage at the University of Edinburgh, and a founding member of the Centre for
Language Evolution. She uses experimental and computational tools to investigate
how the human cognitive system shapes linguistic typology. She received the Robert
J. Gushko Prize for Outstanding Doctoral Dissertations in Cognitive Science in 2012,
was elected to the Young Academy of Europe in 2019, and currently holds a European
Research Council Starting Grant.

Ian Cunnings is an Associate Professor of Psycholinguistics in the School of Psychol-

ogy and Clinical Language Sciences at the University of Reading, UK. His main research
interests are in sentence and discourse-processing in different populations of speakers.
His work has examined the memory-encoding, storage, and retrieval mechanisms that
subserve the resolution of different types of linguistic dependencies during language
comprehension. His most recent research examines how these different memory oper-
ations can inform our understanding of the factors that influence successful sentence
comprehension in native and non-native speaker populations.

Brian Dillon is an Associate Professor in the Department of Linguistics at the Univer-

sity of Massachusetts, Amherst. His research focuses on adult sentence comprehension,
aiming to understand how linguistic constraints are deployed in real-time to constrain
sentence processing. In his work, he integrates insights from linguistic theory with
process-level cognitive models, with a particular interest in the processing of agreement
and pronominal reference.

Stephani Foraker is an Associate Professor in the Department of Psychology at State

University of New York College at Buffalo, specializing in cognition and psycholinguis-
tics. Her main research interests are in sentence and discourse processing, focusing on
the role of memory and focal attention. She has used the speed–accuracy tradeoff (SAT)
procedure to investigate long-distance dependencies and pronoun resolution, training
under Brian McElree, who pioneered the application of SAT to psycholinguistic issues.
Her current research examines the contribution of hand gestures as part of encoding,
storage, and retrieval operations, particularly in anaphora resolution.

Tim Hunter is an Associate Professor in the Department of Linguistics at the Univer-

sity of California, Los Angeles. The bulk of his research uses computational perspectives
to investigate the formal properties of natural language grammar, with one main goal
being to clarify the consequences of taking linguistic theories to be testable cogni-
tive hypothesis. This line of work includes studies connecting minimalist syntax to
experimental work in language-processing, and studies of the relationship between de-
terminers’ truth-conditions and verification procedures. He has also worked on the
argument/adjunct distinction and the syntax of ellipsis.

Elsi Kaiser is a Professor in the Department of Linguistics at the University of Southern

California. She received her PhD in linguistics from the University of Pennsylvania, af-
ter a BA in Germanic languages and literatures from Princeton University and an MA in
 the contributors xix

psychology from the University of Pennsylvania. Her research focuses on the processes
and representations involved in comprehension and production, especially in domains
involving multiple aspects of linguistic representation (syntax, semantics, pragmatics),
such as reference resolution. She has investigated multiple languages (e.g. Finnish, Es-
tonian, French, German, and Dutch, including collaborative work on Bangla/Bengali,
Hindi, Italian, Korean, Chinese, and Vietnamese).
Dave Kush is an Assistant Professor of Linguistics at University of Toronto. He is
interested in sentence-processing, syntactic theory, and cross-linguistic variation.
Jeffrey Lidz is Distinguished Scholar-Teacher and Professor of Linguistics at the Uni-
versity of Maryland. His research explores language acquisition from the perspective
of comparative syntax and semantics, focusing on the relative contributions of experi-
ence, extralinguistic cognition, and domain-specific knowledge in learners’ discovery
of linguistic structure and linguistic meaning.
Andrea E. Martin is a Lise Meitner Research Group Leader at the Max Planck Institute
for Psycholinguistics, and a Principal Investigator at the Donders Centre for Cogni-
tive Neuroimaging at Radboud University in Nijmegen, the Netherlands. Her work
has spanned structural and semantic aspects of sentence processing. She has used the
speed–accuracy trade-off procedure and cognitive neuroimaging to study the role of
memory in sentence processing via ellipsis, a line of research begun with Brian McElree,
who pioneered application of SAT to psycholinguistic issues. The current focus of her
lab, Language and Computation in Neural Systems, is on developing theories and mod-
els of language representation and processing which harness the computational power
of neural oscillations, such that formal properties (viz., constituency, compositionality)
can be realized in biological and artificial neural networks.
William Matchin is an Assistant Professor of Communication Sciences and Disorders
in the Arnold School of Public Health at the University of South Carolina. As part of
the Center for the Study of Aphasia Recovery, he directs the NeuroSyntax lab, using
functional neuroimaging and lesion–symptom mapping and incorporating insights of
linguistic theory to understand the architecture of language in the brain. He is currently
investigating the nature of grammatical deficits in aphasia, including paragrammatism
and agrammatism.
Lisa S. Pearl is a Professor in the Department of Language Science at the University
of California, Irvine. Her research lies at the interface of language development, com-
putation, and information extraction, including both cognitively oriented research and
applied linguistic research that combines theoretical and computational methods. Her
cognitively oriented research focuses on child language acquisition, with a particular
focus on theory evaluation via acquisition-modeling, and how children’s input affects
their linguistic development.
Laurel Perkins is an Assistant Professor in the Department of Linguistics at the Uni-
versity of California, Los Angeles. She earned her PhD in linguistics from the University
xx the contributors

of Maryland and held a postdoctoral fellowship in the Laboratoire de Sciences Cogni-

tives et Psycholinguistique at the École Normale Supérieure. Her research studies the
earliest stages of syntax acquisition in infancy, drawing from formal linguistics, devel-
opmental psychology, and computational cognitive modelling. She is a recipient of a
Post-Doctoral Study Grant from the Fyssen Foundation, a Doctoral Dissertation Im-
provement Grant from the National Science Foundation, and a Glushko Dissertation
Prize from the Cognitive Science Society.

Maria Polinsky is Professor of Linguistics, Associate Director of the Language Science

Center, and Director of Research Field Stations at the University of Maryland. She has
conducted extensive primary work on several languages of the Caucasus, Austronesian
languages, and Chukchi. She is also engaged in a comprehensive research program on
heritage languages. Her work emphasizes the importance of lesser-studied languages
for theoretical linguistics. Recent publications include Deconstructing Ergativity (2016),
Heritage Languages and Their Speakers (2018), and The Oxford Handbook of Languages
of the Caucasus (2021).

Corianne Rogalsky is an Associate Professor of Speech and Hearing Science in the

College of Health Solutions at Arizona State University (ASU). As Director of ASU’s
Communication Neuroscience and Neuroimaging Lab, Rogalsky uses behavioral and
neuroimaging techniques to better understand the neural and cognitive resources that
support effective communication in everyday life for individuals who have experi-
enced a brain injury such as a stroke. Rogalsky’s current focus is investigating how
executive functions such as selective attention and working memory support speech
comprehension in neurotypical adults, and how that support may change after a stroke.

Jeffrey Runner is a Professor of Linguistics and Brain & Cognitive Sciences, Dean of
the College, and Vice Provost and University Dean for Undergraduate Education at
the University of Rochester. He earned a BA in linguistics at the University of Califor-
nia, Santa Cruz, in 1989 and a PhD in linguistics at the University of Massachusetts at
Amherst in 1995. He joined the department of Linguistics at the University of Rochester
in 1994. His research uses experimental methodologies to investigate natural language
syntax. In 2017, he became dean of the College in Arts, Sciences and Engineering,
and is responsible for the curricular, co-curricular, and extra-curricular undergraduate
experience.

Jon Sprouse is a Professor of Psychology at New York University Abu Dhabi. He re-
ceived an AB in linguistics from Princeton University (2003) and a PhD in linguistics
from the University of Maryland (2007). His research focuses on the use of experi-
mental syntax techniques, including acceptability judgments, EEG, and computational
modeling, to explore fundamental questions in syntax. He has authored over forty jour-
nal articles and book chapters on experimental syntax. His work has been recognized
by the Best Paper in Language award, the Early Career award, and the C. L. Baker
mid-career award from the Linguistic Society of America.
 the contributors xxi

Kristen Syrett is an Associate Professor in the Department of Linguistics at Rutgers, the

State University of New Jersey–New Brunswick, with a co-appointment at the Center
for Cognitive Science (RuCCS). She is the Director of the Laboratory for Developmental
Language Studies. Her research focuses on semantics and its interface with pragmatics
and syntax in language acquisition and development, and on experimental semantics
and pragmatics in adult psycholinguistics.
Kriszta Eszter Szendrői is a Professor of Theoretical and Experimental Linguistics at
the University of Vienna. She works on information structure, including its syntax and
prosody, using both theoretical and experimental means, working with both adults
and children. She has also worked on the syntax of scope, especially on the acquisi-
tion of scope. She is also interested in the interactions between information structure
and scope. On a different note, for the past few years she has been leading a research
project studying the grammar of Contemporary Hasidic Yiddish.
Matthew Wagers is Professor of Linguistics at the University of California, Santa
Cruz, where he has taught since 2009. The focus of his research is how syntactic in-
formation is represented in memory and how morphological cues guide incremental
interpretation. These are cross-cut by an interest in broadening the contribution of psy-
cholinguistically under-investigated languages to theory development. He holds a PhD
in linguistics from the University of Maryland (2008) and an AB in Molecular Biology
from Princeton University (2003).
Masaya Yoshida is an Associate Professor in the Department of Linguistics at North-
western University. Research interests include online sentence processing and syntax.
He has worked on the syntax and processing of ellipsis constructions, long-distance
dependencies, and islands. Some of his recent studies have explored the structure as-
sociated with the ellipsis site in clausal ellipsis constructions, and how structure in the
ellipsis site is built during online sentence-processing.
 pa rt i
...................................................................................................

JUDGMENT
METHODS IN
S Y N TA C T I C
T H E O RY
...................................................................................................
 c ha p t e r 1
...........................................................................................................

a c c e p ta b i l i t y
judgments
...........................................................................................................

jon sprouse

1.1 Introduction
..........................................................................................................................

The goal of experimental syntax, at least to my mind, is straightforward: to use exper-

imental methods to collect data that is relevant for the construction and evaluation of
syntactic theories. For data types that can only be collected using a formal experiment,
such as reaction times or EEG, the work of experimental syntax is simply the work of
leveraging these methods for questions in theoretical syntax. However, things appear to
be a bit more complicated when the data type in question is acceptability judgments, as
acceptability judgments can be collected both relatively informally, as is typical in much
of the syntax literature, or relatively formally, as is typical in the experimental syntax
literature. I take the coexistence of these two methods of judgment collection to imply
that the goal of experimental syntax with respect to acceptability judgments is not sim-
ply to collect acceptability judgments, because that is what is done in all syntactic work,
but rather to explore ways in which the formal collection of judgments can add new in-
sights over and above those that derive from informal methods. Therefore, my goal in
this chapter is to identify four areas in which formal judgment experiments have made
substantial contributions—two that lean toward methodological issues, and two that
lean toward theoretical issues—and to review the current state of the evidence that we
have for each of those areas. To be clear, this chapter is not intended as an exhaustive re-
view of all possible areas in which formal judgment experiments could potentially make
a contribution; rather, it is intended as a starting point for thinking about the kinds of
questions in theoretical syntax that might benefit from formal acceptability judgment
experiments. My hope is that these questions will help to inspire new questions, and
new work, in the growing field of experimental syntax.
4 jon sprouse

Before delving into the primary content of this chapter, I would like to briefly men-
tion a few assumptions (and/or decisions) that I am making. The first is that I assume,
following many working syntacticians, that acceptability judgments are in principle
valuable for the construction and evaluation of syntactic theories. I will, therefore, not
attempt to motivate the use of acceptability judgments in general (see Schütze 1996
for a comprehensive discussion of this). The second is that I will assume a relatively
minimal linking hypothesis between acceptability judgments and the cognitive prop-
erties of sentence processing. Under this linking hypothesis, an acceptability judgment
is a relatively automatic behavioral response that arises when a speaker comprehends
a sentence, and that this behavioral response is impacted by a large number of cog-
nitive factors, such as the grammaticality of the sentence, the processing dynamics of
the sentence, the sentence-processing resources required by the sentence, the meaning
of the sentence, the plausibility of the sentence relative to the real world, and even the
properties of the specific task that is given to the speaker. I believe wholeheartedly that
a more precise linking hypothesis would be helpful for using judgments as evidence in
syntax; however, I also believe that the minimal linking hypothesis above is more than
sufficient to begin to explore the value of formal acceptability judgment experiments
in syntax. My third assumption is that there is no substantive qualitative difference
between “informal” and “formal” judgment experiments. Both are experiments in the
sense that they involve the manipulation of one variable (syntactic structure) to re-
veal a causal relationship with another variable (acceptability). Therefore both involve
all of the components that typify psychology experiments: a set of conditions, a set of
items in each condition, a set of participants, a task for the participants to complete
using the items, and a process for analyzing the results of the task. The difference ap-
pears to me to be primarily quantitative, in that “formal” experiments tend to involve
more conditions, more items per condition, more participants, and more complex anal-
ysis processes. To my mind, the labels “informal” and “formal” simply point toward
different ends of this quantitative spectrum. In practice, when I say that formal ex-
periments are valuable in some way, what I mean is that increasing the number of
conditions, items, or participants, and/or increasing the complexity of the analysis,
can yield insights that fewer conditions, items, participants, and/or less complex anal-
yses cannot. The labels “informal” and “formal” are a more concise way to express
this idea. My fourth assumption is that Schütze 1996 already provides a comprehen-
sive review of experimental syntax work that was published before 1996. Therefore,
in order to provide something new for the field, I will focus here on work published
after 1996.
Finally, this chapter is not a how-to for constructing formal judgment experiments.
The goal is for this to be the chapter one reads, either before or after reading a how-
to, for inspiration about the types of questions one can ask with the method. I will
provide some references for learning acceptability judgment methods in the annotated
bibliography for Part I of this Handbook.
 acceptability judgments 5

1.2 The validity and reliability of

acceptability judgments
..........................................................................................................................

Perhaps the most frequently asked question in the experimental syntax literature is to
what extent the informally collected judgments that have been published in the liter-
ature can be trusted to form the empirical basis of syntactic theory. This question has
arisen since the earliest days of generative grammar (Hill 1961; Spencer 1973); it played
a central role in the two books that ushered in the most recent wave of interest in ex-
perimental syntax (Schütze 1996; Cowart 1997); and it has given rise to a number of
high-level debates in the experimental syntax literature over the past 15 years or so
(see Edelman and Christiansen 2003; Ferreira 2005; Wasow and Arnold 2005; Feather-
ston 2007; Gibson and Fedorenko 2013 for some concerns about informal methods; see
Marantz 2005 and Phillips 2009 for some rebuttals, and Myers 2009 for a proposal that
attempts to split the difference between informally collected judgments and full-scale
formal experiments). The existence of this question is understandable. First, informally
collected judgments form the vast majority of the data points published in the (gener-
ative) syntax literature. Second, the properties of informal collection methods are not
identical to the properties of the formal experimental methods that are often used in
other domains of cognitive science: Informal methods often involve a smaller num-
ber of participants, those participants are often professional linguists instead of naïve
participants, the participants are often presented a smaller number of items, and the re-
sults are often only analyzed descriptively (without inferential statistics). If one believes
that the properties of formal experiments are what they are to ensure the quality of the
data, then it is logically possible that the differences between informal methods and for-
mal experiments could lead to lower-quality data. The consequences of this cannot be
understated. If there are systemic problems with informally collected judgments, then
there are likely to be systemic problems with (generative) syntactic theories.
This question touches upon a number of issues in psychometrics and the broader
philosophy of measurement. The first question is: What do we mean when we say that
data can be “trusted” to form the basis for a theory? Psychometric theories have iden-
tified a number of properties that good measurement methods should have. Here I will
mention two (and only in a coarse-grained way, setting aside subtypes of these proper-
ties): validity and reliability. A measurement method is valid if it measures the property
it is purported to measure. A measurement method is reliable if it yields consistent re-
sults under repeated measurements (with unchanged conditions). The concerns about
informal methods that have figured most prominently in the literature appear to be a
combination of concerns about validity and reliability, such as the concern that small
sample sizes will lead to an undue influence of random variation, the concern that a
small number of experimental items will lead to an undue influence of lexical proper-
ties, and the concern that the participation of professional linguists will lead to theo-
retical bias. In each case, the concern seems to be that informally collected judgments
6 jon sprouse

will not reflect the true acceptability of the sentence (validity), and furthermore that the
judgments themselves will be inconsistent over repeated measurements (reliability).
This leads to a second question: How does one establish validity for the measurement
of a cognitive property like acceptability? The direct method for establishing validity is
to compare the results of the measurement method with a second, previously validated,
measurement method. This is obviously unavailable for most cognitive properties—if
cognitive scientists had a method to directly measure the cognitive property of interest,
we would not bother with the unvalidated measurement method. That leaves only indi-
rect methods of validation. One indirect method is to ask whether the theory that results
from the data has the properties of a good scientific theory. This, of course, interacts
with broader issues in the philosophy of science about what properties a good theory
would have, so I will not attempt to provide an exhaustive list. But two possible criteria
are: (i) making potentially falsifiable predictions, and (ii) explaining multiple phenom-
ena with a relatively small number of theoretical constructs. In the case of acceptability
judgments, I would argue that the resulting theory of syntax does, indeed, have these
properties. Another indirect method is to ask whether other data types provide corrobo-
rating evidence, modulo the linking theories between the data types and the underlying
theory. In the case of acceptability judgments, we can ask whether the resulting syntac-
tic theory can be linked to a sentence-processing theory in a way that makes potentially
falsifiable predictions about other psycholinguistic measures, such as reading times, eye
movements, or EEG, and ultimately whether these measures corroborate the syntactic
theory. I would argue that the current results in the literature connecting syntactic theo-
ries and sentence-processing theories are promising. That said, indirect methods cannot
guarantee validity. It is logically possible that acceptability judgments could give rise to
a theory that has all of the hallmarks of a good theory, but that does not ultimately
explain human syntax (perhaps the resulting theory is actually about probability, or
plausibility, or even prescriptive grammatical rules).
This leads to the final question: How does one establish reliability? In principle, estab-
lishing reliability is relatively straightforward, as it simply entails replicating the mea-
surement. The exact replication can vary based on the type of reliability one is interested
in: Between-participant (or inter-rater) reliability asks whether the same judgments are
obtained with different sets of participants; within-participant (or test–retest) reliability
asks whether one set of participants will give the same judgments at two different times;
between-task reliability asks whether different judgment tasks will yield the same judg-
ments (either between-participant or within-participant). In practice, establishing the
reliability of informal methods is complicated by their informality. By definition, in-
formal methods control the various properties of the judgment collection process less
strictly than formal methods, making a strict replication difficult if not impossible. One
way to circumvent this problem is to compare the results of informal methods, perhaps
as reported in the syntactic literature, with the results of formal experiments. This would
be a type of between-task reliability for informal and formal methods, and to the extent
that the two sets of results converge, it would establish a kind of reliability for informal
methods. Many of the results reported below test precisely this kind of reliability. But
Another random document with
no related content on Scribd:
254

Smell and taste, perversion of,

250

Spinal irritation,

251

Temperature and pulse,

252

Urinary secretion, state of,

253

Vaginismus,

246

Vomiting,

254
 Synonyms,

206

Treatment,

273

Baths, use of, in,

282

283

Bromides, use of,

276

Chloroform, use of,

285

286

Climatic,
283

Cold, use of,

282

283

Education, necessity of proper,

274

Electricity, use of,

281-286

Emetics in controlling seizures,

285

Exercise and gymnastics, use in,

275

280
 Gold and sodium chloride, use of,

279

Gynæcological, question of,

286

287

Harsh measures, questionable value of,

276

Hydrotherapy in,

281

282

in children,

275
 Iron and zinc salts, use in,

278

Massage and Swedish movements,

280

Metallo-therapy,

284

Mind- and faith-cures, value of,

277

278

Mitchell's rest-cure,

279

Moral,

276
 Musk, valerian, asafœtida, etc.,

278

Nitrite of amyl,

285

of contractures,

286

of paralyses,

286

of paroxysms,

285

Opium, use of,

278

Oöphorectomy, question of,

287
 Prophylactic and hygienic,

274

Sea-bathing in,

283

Hysteria as a cause of disseminated sclerosis,

884

of vertigo,

425

distinguished from brain tumors,

1055

Hysterical headache,

402
 insanity,

148

YSTERO-EPILEPSY

288

Definition and synonyms,

288

Diagnosis,

307

from true epilepsy,

308

from simulation,
310

Duration and course,

307

Etiology,

291

Age, influence of,

291

Emotional,

293

Painful menstruation,

293

Sex, race, and climate,

291
 History,

289

Pathology,

291

Prognosis,

310

Symptoms,

293

Anæsthesia,

298

Contracture,

297

Digestive,

297
 Hallucinations,

301

Hystero-epileptogenic zones, significance of,

298

of contortions and great movements,

301

of delirium,

301

of emotional attitudes,

301

of epileptoid period,

300

of irregular types,
302

of regular types,

293

of paroxysms,

293

304

306

Ovarian hyperæsthesia,

298

pressure, effect of,

299

Permanent,

307
 Prodromal,

297

Treatment,

310

Compression of nerve-trunks,

310

Electricity, use of,

311-313

Metallic salts, use of,

313

Metallo-therapy,

313

Nitrite of amyl and nitro-glycerin,

311

Oöphorectomy, question of,

312

Potassium bromide, use of,

313

Torsion of abdominal walls to arrest paroxysms,

311

Varieties,

290

I.

Ice, use of, in acute simple meningitis,

720
 in cerebral meningeal hemorrhage,

715

in neuritis,

1194

in tubercular meningitis,

736

Idiocy, intellectual and moral,

138

Ill-health, influence on causation of insanity,

116

117
Illusions in nervous diseases,

20

Imbecility, intellectual and moral,

138

in inflammation of the brain,

791

Imitation, influence on causation of catalepsy,

319

of hysteria,

222-229

Impotence in tumors of spinal cord,

1096

Impulsive insanity,

146

Inco-ordination in nervous diseases,

47-50

Inebriety, trance state in,

346

NFANTILE

PINAL

P
ARALYSIS

1113

Anatomical lesions,

1131

Atrophy of anterior spinal horns, characters, etc.,

1133

1138

1139

nerve-roots and anterior columns of the cord,

1138

1139

Autopsies, tables of,

1133-1136

1139

Microscopic lesions,

1137

Theories interpreting the lesions of the cord,

1140-1444

regarding origin, premature,

1132

Complication with progressive muscular atrophy,

1149

Course of,

1148

Definitions,
1113

Deformities of atrophic paralysis,

1127

Criticism of theory of muscular antagonism,

1128

Dislocations in,

1130

Mechanism of,

1128

1131

Muscular contraction in,

1127

Relation of weight and muscular forces,